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1 .  Project  Objectives 


In  the  21®‘  century,  the  fundamental  paradigm  of  computing  will  shift  to  one  of 
intelligent,  distributed  agents,  which  are  capable  of  efficiently  processing  asynchronous 
input  from  humans  and  from  other  computational  agents,  and  of  producing  timely  output. 
The  intelligent  distributed-agents  paradigm  will  be  especially  valuable  for  a  range  of 
military  applications  that  involve  the  processing  of  massive  amounts  of  heterogeneous 
information,  interrelated  in  complex  ways,  in  time-critical  situations.  Such  agent-based 
systems  have  four  key  characteristics.  First,  they  receive  inputs  asynchronously,  with 
additional  input  often  arriving  before  the  processing  of  previous  inputs  is  complete. 
Second,  the  outputs  of  these  systems  may  not  be  immediate,  and  so  commitments  to 
future  events  and  reservations  for  future  resources  must  be  made.  Third,  there  may  be 
multiple  ways  of  processing  any  input,  where  the  decision  among  these  depends  on  the 
commitments  and  reservations  already  made.  Fourth,  each  system  may  be  part  of  a  larger 
“community”  of  systems,  so  that  issues  of  coordination,  competition,  and  communication 
come  into  play. 

The  goal  of  this  project  was  to  investigate  the  foundations  of  systems  with  these  four 
characteristics,  aiming  towards  the  objective  of  developing  a  predictive  theories  of  agent- 
system  behavior  that  are  useful  both  for  designing  agent  systems  and  for  predicting  the 
performance  of  agent  systems  that  are  inherently  ill-suited  for  full-fledged  realistic 
testing  (e.g,  systems  whose  misperformance  could  have  disastrous  effects).  In  particular, 
in  this  project  we  focused  on  one  critical  aspect  of  agent-based  systems:  commitment 
strategies,  which  agent-based  systems  must  employ  to  cope  with  asynchronous  input, 
deferred  output,  and  the  significant  demands  on  the  computational  processing  that 
necessitate  the  careful  management  of  computational  resources.  Commitments  can  be 
viewed  as  guarantees  that  certain  actions — computational  or  external — will  be  performed 
by  the  individual  agent,  in  a  way  that  satisfies  given  execution  constraints  (These 
constraints  may,  for  example,  specify  time  or  duration  of  performance,  or  quality  of 
results  produced). 

To  effectively  manage  a  set  of  commitments,  an  agent  must  be  able  to  perform  the 
following  computations: 

•  determine  whether  a  new  option  for  action  (a  new  processing  request)  requires  the 
abandonment  or  modification  of  any  of  its  existing  commitments; 

•  if  it  doesn’t,  determine  the  most  efficient  way  to  add  a  commitment  to  the  new 
option; 

•  if  it  does,  determine  whether  the  set  of  prior  commitments  should  be  modified, 
and  what  the  minimal-impact  modification  is. 

Our  emphasis  on  this  project  has  been  on  the  first  two  computations:  we  developed  a  set 
of  computationally  efficient  techniques  for  both  determining  the  consistency  of  sets  of 
actions  in  order  to  determine  whether  or  not  newly  introduced  actions  are  compatible 
with  existing  commitments,  and  for  merging  new  commitments  into  sets  of  existing  ones. 
We  also  did  work  on  the  third  topic,  developing  techniques  for  deciding  whether  and  how 
to  modify  a  set  of  commitments  in  response  to  a  new,  incompatible  option  in  a  way  that 
minimizes  a  weighted  function  of  the  changes  on  the  existing  options  and  the  cost  of  the 
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resulting  set  of  commitments.  Additionally,  we  applied  our  algorithms  to  various 
applications  that  were  of  interest  (at  various  times)  to  the  TASK  effort,  including  e- 
commerce,  a  “briefing  agent”,  and  autonomous  unmanned  vehicles;  we  also  leveraged 
this  work  by  using  the  algorithms  developed  in  the  current  project  in  the  development  of 
cognitive  orthotic  systems  (which  were  supported  by  other  projects). 

2.  Research  Results 

The  principal  results  obtained  during  the  project  include  the  following  (annotated  with 
the  publication  in  which  they  are  reported  in  more  detail): 

•  Formulation  of  the  problem  of  commitment  management  as  one  of  temporal 
constraint-satisfaction  processing  (CSP),  and  development  of  a  general 
framework  for  doing  consistency  checking  and  update  using  temporal  CSPs 
[5,6,10]. 

•  Design,  implementation,  and  experimental  assessment  of  an  efficient  algorithm 
for  checking  the  consistency  of  and  updating  tasks  represented  as  disjunctive 
temporal  problems,  demonstrating  speed-up  of  two  orders  of  magnitude  relative 
to  the  previous  state-of-the-art  [1,10]. 

•  Development  of  an  algorithm  for  flexible  dispatch  of  plans  encoded  as  DTPs  that 
has  a  set  of  strong,  provable  properties  [9,1 1,15]. 

•  Formulation  of  a  new  temporal  CSP  representation.  Conditional  Temporal 
Problems  (CTPs)  used  to  represent  tasks  that  include  contingent  outcomes, 
and  development  of  algorithms  for  consistency-checking  and  update  of 
tasks  encoded  as  CTPs  [2,15]. 

•  Discovery  of  an  undetected  incompleteness  in  previous  formalizations  of  the 
planning  with  contingencies,  revealed  by  and  corrected  in  the  CTP  formalism  [2]. 

•  Preliminary  development  of  a  strategy  for  dispatching  plans  encoded  as  simple 
temporal  networks  with  uncertainty  (STP-u’s),  even  when  they  are  not  consistent 

[4]. 

•  Application  of  the  consistency-checking  and  update  algorithms  to  applications 
that  were  of  interest  to  the  TASK  effort,  including  e-commerce,  a  “briefing 
agent”,  and  unmanned  autonomous  vehicles,  as  well  as  to  cognitive  orthotics 
systems  [3,6,7,8,12]. 

•  Development  of  an  effective  algorithm  for  replanning  in  response  to  a  task  that  is 
not  consistent  with  existing  commitments,  so  as  to  maximize  a  weighted  function 
of  the  changes  on  the  existing  options  and  the  cost  of  the  resulting  set  of 
commitments  [13,14]. 

These  main  thrusts  are  very  briefly  described  below. 
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2.1  Commitment  Management  as  Temporal  Constraint-Satisfaction 
Processing 

The  first  task  for  a  commitment-management  component  of  a  system  is  to  determine 
whether  or  not  a  new  option  is  potentially  consistent  with  existing  commitments,  and  if 
so,  what  additional  constraints  need  to  be  made  to  ensure  the  consistency.  The  problem 
can  be  cast  as  one  of  temporal  reasoning,  in  which  the  commitment  set  and  the  new 
option  are  modeled  as  sets  of  temporally  grounded  constraints  on  future  actions,  and  the 
problem  is  to  determine  the  consistency  of  the  union  of  these  sets.  A  side-effect  of  many 
consistency-checking  algorithms  is  the  inference  of  additional  constraints  that  are  needed 
to  ensure  consistency.  A  key  question  is  when  consistency-checking  must  be  performed. 
In  general,  there  are  four  triggers:  the  introduction  of  a  new  action;  a  modification  to  the 
existing  commitments;  the  execution  of  an  action;  or  the  passing  of  a  critical  time 
boundary  (e.g.,  the  latest  start  time  for  one  alternative  way  of  performing  an  existing 
commitment).  By  modeling  commitment  management  as  a  temporal  constraint- 
satisfaction  problem,  we  are  able  both  to  take  advantage  of  known  efficiencies  in  CSP 
processing,  and  to  tailor  the  consistency-checking  algorithm  to  the  particular 
representational  requirements  of  a  given  agent  task-set  (e.g.,  whether  it  requires  the 
inclusion  of  disjunctive  constraints;  the  modeling  of  causal  uncertainty;  and/or  the 
modeling  of  temporal  uncertainty). 

See  references  [5,6,10]. 


2.2  Efficient  Processing  of  Disjunctive  Temporal  Problems 

For  many  years,  consistency  checking  of  sets  of  temporal  constraints  was  limited  to 
classes  of  constraints  with  restrictive  expressiveness,  particularly  what  are  called  Simple 
Temporal  Problems  (STPs).  Although  such  problems  can  be  solved  with  polynomial¬ 
time  shortest-path  algorithms,  they  include  only  atomic  (non-disjunctive)  constraints,  and 
thus  are  not  suitable  for  modeling  advanced  agent-based  systems.  Not  only  does  the 
restriction  to  STPs  preclude  complex  temporal  constraints,  but  it  also  precludes  least- 
commitment  approaches  to  conflict  resolution,  since  expressing  a  constraint  of  the  form 
“Either  do  A  before  B  or  after  C”,  i.e.,  promote  or  demote,  is  not  expressible  in  an  STP. 
To  handle  such  constraints,  one  can  employ  a  richer  formalism:  Disjunctive  Temporal 
Problems  (DTPs).  Checking  the  consistency  of  a  DTP  is  an  NP-hard  problem,  and 
therefore  heuristic  techniques  are  pursued.  We  conducted  a  systematic  analysis  of 
previous  approaches  to  DTP-solving,  and  used  that  as  the  basis  of  a  new  algorithm,  called 
Epilitis,  that  is  two  orders  of  magnitude  faster  than  previous  DTP-solvers.  Epilitis  was 
fully  implemented  and  has  been  made  available  to  the  research  community. 

See  references  [1,10]. 
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2.3  Dispatch  of  Disjunctive  Temporal  Probiems 

By  definition,  agent  systems  need  not  only  to  manage  their  commitments,  but  also  to 
execute  them.  The  dispatch  problem  is  the  problem  of  deciding  when  to  execute  each 
action  to  which  the  agent  has  committed.  When  the  commitments  have  temporal 
constraints,  the  dispatch  problem  may  be  non-trivial.  As  with  consistency-checking, 
most  prior  work  on  dispatch  focused  on  STPs.  We  developed  a  dispatch  algorithm  that  is 
applicable  to  DTPs.  We  identified  a  set  of  properties  that  any  dispatch  algorithm  must 
possess — it  must  be  correct,  deadlock-free,  maximally  flexible,  and  useful — and  we 
proved  that  our  algorithm  has  these  properties. 

See  references  [9,1 1,15]. 


2.4  Formulation  of  Conditional  Temporal  Problems 

STPs  and  DTPs  do  not  directly  allow  the  encoding  of  actions  that  have  uncertain 
outcomes.  Yet  often  agents  must  deal  with  precisely  such  actions.  To  facilitate  this,  we 
developed  a  new  formalism,  called  Conditional  Temporal  Problems  (CTPs).  CTPs  are  an 
extension  of  the  standard  temporal  constraint- satisfaction  problem,  which  (1)  include 
observation  nodes,  and  (2)  attach  labels  to  all  nodes,  indicating  the  situations  in  which  the 
action  they  represent  will  be  performed.  The  extended  framework  thus  supports  the 
modeling  of  conditional  actions.  We  developed  algorithms  for  consistency-checking 
with  CTPs:  importantly,  consistency  checking  in  a  CTP  can  be  achieved  even  while 
allowing  for  decisions  about  the  precise  timing  of  actions  to  be  postponed  until  execution 
time,  thereby  adding  flexibility  and  making  it  possible  to  dynamically  adjust  the 
commitments  in  response  to  observations  made  during  execution.  Our  formulation  of 
CTPs  also  led  to  a  discovery  of  a  previously  undetected  incompleteness  in  the  conditional 
planning  literature:  it  turns  out  that,  even  for  plans  without  explicit  quantitative  temporal 
constraints,  prior  approaches  will  sometimes  deem  a  planning  problem  unsolvable  when 
in  fact  there  is  a  solution  to  it.  The  use  of  the  CTP  formalism  eliminates  this  problem. 

See  reference  [2]. 


2.5  Dispatching  Temporai  Problems  with  Uncertainty 

Where  CTPs  allow  one  to  model  causal  uncertainty,  an  alternative  formalism,  STPU’s 
(Simple  Temporal  Plans  with  Uncertainty)  had  been  proposed  in  the  literature  to  model 
temporal  uncertainty,  and  an  algorithm  for  assessing  the  consistency  of  STPU’s  had  been 
developed.  However,  the  algorithm  provided  no  guidance  to  the  agent  in  the  case  in 
which  the  STPU  could  not  be  guaranteed  to  succeed;  yet,  by  definition,  STPUs  involve 
uncertain  relations,  and  it  may  be  very  likely  that  a  given  STPU  will  be  successfully 
executed  even  if  this  cannot  be  guaranteed.  We  thus  addressed  the  question  of  what  an 
agent  should  do  with  such  an  STPU,  developing  a  set  of  three  alternative  approaches,  one 
based  on  a  binary  search,  one  based  on  iterative  tightening,  and  one  based  on  an 
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approximation  of  linear  optimization.  These  approaches  can  be  used  both  to  provide 
lower  bounds  on  the  probability  of  successful  execution,  and  to  provide  guidance  to  the 
agent  about  when  to  execute  the  actions  involved. 

See  reference  [4]. 


2.6  Applications 

To  demonstrate  the  viability  of  the  representations  and  algorithms  we  developed,  we 
applied  them  to  work  to  various  applications  that  were  of  interest  to  the  TASK  project, 
including  E-commerce,  a  “briefing  agent”,  and  analyses  of  clusters  of  simulated 
unmanned  autonomous  vehicles.  We  also  leveraged  the  work  done  in  the  project  by 
using  the  results  obtained  here  in  work  we  did  with  the  support  of  other  contracts  to 
develop  a  cognitive  orthotic  system,  which  is  a  form  of  a  plan-management  agent  used  to 
assist  people  with  memory  decline  in  carrying  out  their  activities  effectively.  Although 
this  latter  application  is  not  within  the  scope  of  the  current  project,  it  is  worth  noting  that 
we  were  able  to  build  heavily  on  the  techniques  developed  here  in  designing  that  agent. 

See  references  [3,6,7,8,12]. 


2.7  Commitment  Modification 

When  an  agent  determines  that  a  new  option  is  not  consistent  with  its  existing 
commitments,  it  needs  to  decide  whether  and  when  to  modify  those  commitments  to 
allow  adoption  of  the  new  option.  We  developed  an  approach  to  commitment 
modification  that  builds  on  our  previous  work  on  cost-assessment  in  context. 
Specifically,  it  formalizes  the  notion  of  modification  cost — or  amount  of  change  in  a  set 
of  commitments — and  provides  a  way  for  the  agent  to  trade  modification  cost  against 
overall  cost  in  context  of  the  revised  plan.  This  work  is  still  in  progress,  and  we  expect  to 
publish  our  results  within  the  next  six  months. 

See  references  [13,14]. 
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Abstract 


Over  the  past  few  years,  a  new  constraint-based  formalism  for  temporal  reasoning  has  been 
developed  to  represent  and  reason  about  Disjunctive  Temporal  Problems  (DTPs).  The  class  of 
DTPs  is  significantly  more  expressive  than  other  problems  previously  studied  in  constraint- 
based  temporal  reasoning.  In  this  paper  we  present  a  new  algorithm  for  DTP  solving,  called 
EpiHtis,  which  integrates  strategies  for  efficient  DTP  solving  from  the  previous  literature,  in¬ 
cluding  conflict-directed  backjumping,  removal  of  subsumed  variables,  and  semantic  branch¬ 
ing,  and  further  adds  no-good  recording  as  a  central  technique.  We  discuss  the  theoretical  and 
technical  issues  that  arise  in  successfully  integrating  this  range  of  strategies  with  one  another 
and  with  no-good  recording  in  the  context  of  DTP  solving.  Using  an  implementation  of 
EpiHtis,  we  explore  the  effectiveness  of  various  combinations  of  strategies  for  solving  DTPs, 
and  based  on  this  analysis  we  demonstrate  that  EpiHtis  can  achieve  a  nearly  two  order-of- 
magnitude  speed-up  over  the  previously  pubHshed  algorithms  on  benchmark  problems  in  the 
DTP  Hterature. 

1.  Introduction 

Expressive  and  efficient  temporal  reasoning  is  essential  to  a  number  of  areas  in  Artificial  In- 
telHgence  (AI).  Over  the  past  few  years,  a  new  constraint-based  formaHsm  for  temporal  rea¬ 
soning  has  been  developed  to  represent  and  reason  about  Disjunctive  Temporal  Problems 
(DTPs)  [Stergiou  and  Koubarakis  1998;  Armando,  CasteHini  et  al.  1999;  Oddi  and  Cesta  2000]. 
The  class  of  DTPs  is  significantly  more  expressive  than  other  problems  already  studied  in  con¬ 
straint-based  temporal  reasoning.  It  extends  the  weH-known  Simple  Temporal  Problem  (STP) 
[Dechter,  Meiri  et  al.  1991]  by  aHowing  disjunctions  and  the  Temporal  Constraint  Satisfaction 
Problem  (TCSP)  ibid,  by  removing  restrictions  on  the  form  of  allowable  disjunctions. 

FormaHy,  a  Disjunctive  Temporal  Problem  (DTP)  is  a  pair  <Vy  C>  where  lU is  a  set  of 
temporal  variables  and  C  is  a  set  of  constraints  among  the  variables.  Every  constraint  C- .  C  is 
of  the  form: 
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where  in  turn,  each  c-j  is  of  the  form  x  —  j  —  b  \  .  V  and  h  .  P  Constraint  c-j  is  called  the 

yth  disjunct  of  the  kh  constraint.  A  solution  to  a  DTP  is  an  assignment  to  each  variable  in  V 
such  that  all  the  constraints  in  C  are  satisfied.  If  a  DTP  has  at  least  one  solution,  it  is  consistent. 
Notice  that  each  constraint  C-  may  involve  more  than  two  temporal  variables,  in  which  case  it 
is  not  a  binary  constraint.  Only  DTPs,  and  not  STPs  or  TCSPs,  allow  difference  constraints  of 
arbitrary  arity. 

The  increased  expressivity  of  DTPs  makes  them  a  suitable  model  for  many  planning  and 
scheduling  problems.  Plan  generation,  plan  merging,  job-shop  scheduling,  and  even  temporal 
reasoning  under  certain  forms  of  uncertainty  can  aU  be  modeled  as  DTPs.  As  a  motivating 
example,  consider  a  temporal  plan  with  steps  xl  and  B  that  both  represent  actions  requiring  the 
same  unary  resource,  e.g.,  they  both  use  the  same  printer.  In  this  case,  the  executions  of  xl  and 
B  should  not  overlap.  We  can  encode  this  fact  as  a  DTP  constraint  by  defining  the  DTP  vari¬ 
ables  startfA),  start(B)y  end(A.)y  and  end(B)  associated  with  the  instants  of  starting  and  ending  xl 
and  B.  The  DTP  constraint  then  is  the  following: 

end  (A)  -  start  (B)  =  0 .  end  (B)  -  start{A)  -  0 
i.e.  either  xl  finishes  before  B  starts  or  vice  versa.  It  is  easy  to  see  how  such  constraints  can 
encode  classical  threat  resolution  in  planning.  Analysis  of  the  DTP  defining  this  plan  can  reveal 
whether  it  is  feasible:  the  DTP  is  consistent  if  and  only  if  there  is  a  way  to  execute  the  plan 
such  that  the  deadlines  are  aU  met,  and  the  unary  resource  (the  printer)  is  never  used  more  than 
once  at  the  same  time. 

Threat-resolution  constraints  in  planning,  as  just  described,  can  also  be  encoded  as  binary 
constraints  between  intervals.  There  are  situations  however  when  this  is  not  possible  and 
higher  arity  constraints  are  required  and  thus  cannot  be  expressed  (in  the  general  case)  by  any 
other  current  formalism  but  the  DTP.  For  example  consider  reasoning  about  the  following 
scenario:  'df  you  can,  stop  by  the  post-office  for  10-15  minutes,  then  take  route  xl  for  10-15 
minutes,  or  else  take  route  B  for  10-15  minutes.”  If  we  use  PO  to  denote  the  fluent  5n  the 
post-office’,  then  this  scenario  can  be  represented  as  the  following  constraints: 

(10  =  end(PO)  -  start(PO)  =  15  .  10  =  end  {A)  -  start(B)  =  15  .  end(PO)  =  start(A)) . 

10  =  end (B)  -  start(A)  =  15 

In  turn,  this  can  be  directly  converted  to  DTP  form. 

The  principal  approach  to  DTP  solving  taken  in  the  literature  has  been  to  convert  the 
original  problem  to  one  of  selecting  one  disjunct,  x.  —  Xj  =  bj.  from  each  constraint  C-  .  C 
and  then  checking  that  the  set  of  selected  disjuncts  forms  a  consistent  STP.  Checking  the  con¬ 
sistency  of  and  finding  a  solution  to  an  STP  can  be  performed  in  polynomial  time  using  short- 
est-path  algorithms  [Dechter,  Meiri  et  al.  1991].  The  computational  complexity  in  DTP  solving 
derives  from  fact  that  there  are  exponentially  many  sets  of  selected  disjuncts  that  may  need  to 
be  considered;  the  challenge  is  to  find  ways  to  efficiently  explore  the  space  of  disjunct  combi¬ 
nations.  This  has  been  done  by  casting  the  disjunct  selection  problem  as  a  constraint  satisfac¬ 
tion  processing  (CSP)  problem  [Stergiou  and  Koubarakis  2000]  [Oddi  and  Cesta  2000]  or  a 
satisfiability  (SAT)  problem  [Armando,  CasteUini  et  al.  1999]. 


1  As  is  standard  in  the  literature,  in  this  paper  we  wiU  make  the  assumption,  without  loss  of  generality,  that  the  bounding  values 
b  are  integers. 
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In  this  paper,  we  present  a  new  algorithm  and  heuristics  for  DTP  solving,  embodied  in  a 
system  we  caU  Epilid^^  which  integrates  strategies  for  efficient  DTP  solving  from  the  previ¬ 
ous  literature,  including  conflict-directed  backjumping,  removal  of  subsumed  variables,  and 
semantic  branching,  and  further  adds  no-good  recording  [Schiex  and  VerfaiUie  1994]  as  a  cen¬ 
tral  technique.  We  discuss  the  theoretical  and  technical  issues  that  arise  in  successfully  integrat¬ 
ing  the  previous  strategies  with  one  another  and  with  no-good  recording.  Using  an  implemen¬ 
tation  of  Epilitis,  we  explore  the  effectiveness  of  various  combinations  of  strategies  for  solving 
DTPs,  and  based  on  this  analysis,  we  demonstrate  that  Epilitis  achieves  a  nearly  two-order-of- 
magnitude  speed-up  over  the  previously  published  algorithms  on  benchmark  problems  from 
the  DTP  literature.  This  result  is  based  on  speed  comparisons  because  we  demonstrate  that 
counting  the  number  of  forward-checks  (also  called  consistency-checks),  as  is  commonly  done 
in  the  literature,  is  not  an  accurately  descriptive  measure  of  performance. 

The  overall  structure  of  the  paper  is  as  follows.  In  Section  2  we  present  necessary  back¬ 
ground  information  on  the  DTP  and  DTP  solving.  Section  3  explains  aU  previous  methods  for 
pruning  the  search  for  a  DTP  solution.  Section  4  presents  necessary  background  information 
on  no-good  recording.  Section  5  uses  the  background  material  presented  to  describe  the 
Epilitis  system  and  the  underlying  algorithms  we  used.  In  Section  6  we  present  our  experi¬ 
ments  with  Epilitis.  Section  7  reviews  related  work  in  the  field.  Section  8  concludes  the  paper 
with  an  overall  discussion  and  presentation  of  future  work. 

2.  Solving  Disjunctive  Temporal  Problems 

2.1  The  Basic  Approach 

We  begin  by  reviewing  a  simpler  class  of  temporal  problems:  the  Simple  Temporal  Prob¬ 
lems  (STPs).  An  STP,  like  a  DTP,  is  a  pair  <Vy  C>,  where  V is  again  a  set  of  temporal  vari¬ 
ables;  for  an  STP,  however,  C  is  a  single  constraint  of  the  form  x  —j  =  b,  where  xj  .  V.  Be¬ 
cause  an  STP  contains  only  binary  constraints,  it  can  be  represented  with  a  weighted  graph 
called  a  Simple  Temporal  Network  (STN),  in  which  an  edge  (jy  x)  with  weight  exists  be¬ 
tween  two  nodes  iff  there  is  a  constraint  {x  —j  =  b^^ .  C  .  Polynomial-time  algorithms  can  be 
used  to  compute  the  aU-pairs  shortest  path  matrix,  or  distance  array  of  the  STN.  We  denote  the 
distance  (shortest  path)  between  two  nodes  x  and  j  as  .  The  concept  of  the  distance  is  im¬ 
portant  because  in  a  consistent  STP,  d^  is  the  largest  number  for  which  the  constraint  j  —  x  = 
d^  holds  in  every  solution.  In  addition,  an  STP  is  consistent  if  and  only  if  for  every  node  x  in 
its  associated  STN^,  d^^—  0,  which  means  that  there  are  no  negative  cycles  [Dechter,  Meiri  et  al. 
1991]. 

A  DTP  can  be  viewed  as  encoding  a  collection  of  alternative  STPs.  To  see  this,  recall  that 
each  constraint  in  a  DTP  is  a  disjunction  of  (one  or  more)  STP-style  inequalities.  Let  c-j  be  the 
f  disjunct  of  the  f  constraint  of  the  DTP.  If  we  select  one  disjunct  from  each  constraint  Q 
then  the  set  of  selected  disjuncts  forms  an  STP,  which  we  wiU  call  a  component  STP  of  the  given 
DTP.  It  is  easy  see  that  a  DTP  D  is  consistent  if  and  only  if  it  contains  at  least  one  consistent 


2  From  the  Greek  word  EmXoirjg  (solver). 
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component  STP.  Moreover,  any  solution  to  a  consistent  component  STP  of  D  is  also  a  solu¬ 
tion  to  D  itself  Because  only  polynomial  time  is  required  both  to  check  the  consistency  of  an 
STP,  and,  if  consistent,  extract  a  solution  of  it,  in  the  remainder  of  this  paper  we  will  say  that 
the  solution  of  a  given  DTP  is  anj  consistent  component  STP  of  it.  When  we  need  to  refer  to  an  actual 
assignment  of  numbers  to  the  time  points  in  the  DTP,  we  will  call  this  an  exact  solution.  A  con¬ 
sistent  component  STP  represents  a  number  of  exact  DTP  solutions.  This  is  particularly  im¬ 
portant  in  planning  since  it  provides  execution  flexibility.  The  consistent  component  STP  can 
then  be  executed  as  described  in  [Tsamardinos  1998]. 


Definition  1:  A  time  assignment  to  the  time-points  of  a  DTP  is  called  an  exact  solu¬ 
tion  of  the  DTP.  A  consistent  component  of  a  DTP  is  called  a  solution  of  the  DTP. 


Original  CSP  (the  DTP) 

Meta-CSP 

“Variables” 

x,j,^... 

(Time  Points) 

One  variable  G  for  each  constraint  G  of  the 
original  DTP 

(V  ariables) 

Domains 

(-8  ,  +8 )  for  aU  variables 

D(CiJ  {r//  ^  . .  .^  Ci;^  } 

(Sets  of  Constraints) 

“Constraints” 

-Vl  .  -yn 

i*e  Cn . Cin 

(Disjunctions  of  Constraints) 

Implicitly  defined  by  the  underlying  se¬ 
mantics  of  the  values  in  each  domain. 

Table  1:  Correspondence  between  the  DTP  and  the  meta-CSP 

AU  existing  algorithms  for  DTP  solving,  including  the  one  we  present  in  this  paper,  work 
by  searching  for  a  consistent  component  STP  S  from  a  given  DTP  D  rather  than  attempting  to 
search  directly  for  a  consistent  assignment  to  the  nodes  of  D.  The  process  of  flnding  S  can 
itself  be  modeled  as  one  of  constraint  satisfaction  processing.  Because  the  original  DTP  is  it¬ 
self  also  a  CSP  problem,  we  will  refer  to  the  problem  of  extracting  a  consistent  component 
STP  as  the  meta-CSP  problem.  The  meta-CSP  contains  one  variable  for  each  constraint  C-in  the 
DTP.  The  domain  of  C-  is  the  set  of  disjuncts  in  the  original  DTP  constraint  C- .  The  con¬ 
straints  in  the  meta-CSP  are  not  given  explicitly,  but  must  be  inferred:  an  assignment  satisfles 
the  meta-CSP  constraints  iff  the  assignment  corresponds  to  a  component  STP  that  is  consis¬ 
tent.  For  instance,  if  the  variable  C-  is  assigned  the  value  x  —  j  =  5  it  would  be  inconsistent  to 
extend  that  assignment  so  that  some  other  variable  is  assigned  the  value  j  —  x  —  -G. 

In  this  paper,  we  wiU  refer  to  the  variables  of  the  DTP  as  time  points  and  wiU  reserve  the 
term  variables  for  the  meta-CSP.  We  wiU  use  the  terms  constraint  2ind  ra/we  interchangeably, 
to  refer  to  a  single,  non-disjunctive  constraint  eg.  such  constraints  (values)  constitute  the  do¬ 
mains  of  the  meta-CSP  variables.  Finally,  we  will  reserve  term  node  to  refer  to  the  nodes  of  a 
CSP  tree  search,  which  we  will  typically  be  performing  for  the  meta-CSP — recall  that  we  do 
not  perform  direct  CSP  processing  on  the  DTP.  The  relationship  between  the  original  CSP 
(the  DTP)  and  the  meta-CSP  (which  aims  to  And  a  consistent  component  STP)  is  summarized 
in  Table  1. 

A  typical  forward-checking  CSP  algorithm,  shown  in  Figure  1  can  be  used  to  solve  a 
DTP — or  more  precisely,  to  solve  its  meta-CSP.  The  algorithm  takes  two  parameters:  M,  de- 
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Basic-DTP(^,  U) 

1 .  If  t/=— 1  stop  and  report  A  as  a  solution. 

2.  C.  select-variable(LO,  f/'.  U-{C} 

3.  For  each  value  c  of  d(C)  in  some  order 

4.  A=A.  {C.  c} 

5.  If  forward-check(^ ;  t/ ) 

6.  Basic-DTP(^  ,  t/ ) 

7.  Endlf 

8.  un-forward(t/') 

9.  EndFor 

10.  Rotam  failure 

forward-check(^,  U) 

11.  For  each  variable  CinU 

12.  For  each  value  c  in  d(C) 

13.  If  not  STP-consistency-check(^  .  {C.  c}) 

1 4 .  Remove  c  from  d(C) 

15.  lfd(C)  =  ^ 

16.  xQtnm  false 

17.  Endlf 

18.  Endlf 

19.  EndFor 

20.  Return  true 

Figure  1:  The  Basic  DTP  algorithm 

noting  the  set  of  already  assigned  variables  and  their  assigned  values,  and  U,  the  set  of  as-yet 
unassigned  variables.  The  initial  call  to  solve  a  DTP  C>  should  be  made  with^  =  — i  and 
U  —C  ^  and  the  initial  current  domains  d(C)  should  be  initiahzed  to  the  original  domains 
D(Cf  i.e.  to  the  set  of  constraints  that  constitute  the  disjuncts  in  C-  in  the  DTP  (see  Table  1). 

The  function  select-variable  heuristically  selects  the  next  variable  to  which  to  make  an  as¬ 
signment;  the  decision  about  how  to  make  that  selection  is  left  unspecified  in  the  generic  algo¬ 
rithm,  but  we  discuss  it  further  in  Section  6.4.  The  function  forward-check(^,  U)  performs 
forward  checking,  i.e.,  it  removes  from  the  domains  of  the  variables  still  in  U  all  those  values 
that  are  inconsistent  with  the  current  assignment^,  returning if,  as  a  result,  one  or  more 
variables  in  U  has  a  domain  reduced  to  — i.  Note  that  in  DTP-solving,  forward-check^  oper¬ 
ates  by  checking  the  consistency  of  an  STP  (specifically,  a  component  STP  of  the  DTP), 
which,  as  mentioned  above,  requires  only  polynomial  time.  If  forward-checking  fails,  then  the 
function  un-forward  restores  the  domains  of  the  variables  to  those  before  the  last  call  to  for¬ 
ward-check. 

2.2  Improved  Forward-Checking 

A  large  portion  of  the  computation  in  algorithm  Basic-DTP  is  spent  at  Hne  13  in  forward- 
check,  where  each  value  of  each  variable  is  added  to  the  set  of  constraints  of  the  current  as- 

3  We  only  describe  DTP  solvers  using  forward-check  because  it  has  been  proven  very  efficient  in  DTP  literature  and  it  is  a 
standard  component  of  every  DTP  solver.  In  addition,  not  using  forward-checking  dramaficaUy  reduced  efficiency  in  pre¬ 
liminary  experiments  in  our  lab. 
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forward-check-with-FC(A,  U) 

12. 

For  each  variable  C  in  U 

13. 

For  each  value  c  :x-y  =  bxy  in  d(C) 

14. 

If  b^y  + 

distance  (y,  x,  S) 

15. 

Remove  c  from  d(C) 

16. 

lfd(C)  =  ^ 

17. 

return  false 

18. 

Endlf 

19. 

Endlf 

20. 

EndFor 

21. 

Return  true 

Figure  2:  Forward  checking  in  S TPs  using  the  FC  condition 


signment  ^  and  checked  for  STP-consistency.  STP-consistency  checking  takes  time  0(|F|^) 
where  |F|is  the  number  of  time-points;  thus  forward-check  takes  time  0(v|F|  on  each  node, 
where  v  is  the  number  of  values  to  forward  check.  Fortunately,  there  is  a  computationally  less 
costly  way  of  achieving  forward  checking  of  values,  based  on  the  following  theorem^ 

Theorem  1:  A  value  c-  :j  —x  =  is  inconsistent  with  a  consistent  STP  S  (that  is,  S  . 

c-jis  inconsistent)  if  and  only  if  the  following  condition  holds: 

+  dy^{S)  <  0  (FC-condition) 

where  dy^(S)  is  the  distance  between  nodes  j  and  xin  S. 

Theorem  1  (the  proof  is  in  Appendix  A)  indicates  that  to  forward-check  a  particular  value  j 
— x  =  against  an  assignment  we  just  need  to  check  the  FC-condition.  In  turn,  this  requires 
calculating  the  distances  ^^in  STP  S  for  aU  nodes  x  andj/.  One  method  for  calculating  aU  these 
distances  efficiently  is  to  calculate  the  distance  array;  this  is  equivalent  to  running  fuU  path  con¬ 
sistency,  which  has  time  complexity  time  0(|F|  .  Once  the  distance  array  has  been  calculated, 

the  distance  between  any  two  nodes  j  and  x  can  be  recovered  by  matrix  lookup  in  constant 
time;  hence  the  overall  time  required  for  each  node  is  0(|F|^  +v),  where  v  is  the  number  of 
values  to  be  forward  checked.  An  alternative  technique  is  to  compute  directional  path  consis¬ 
tency  [Chleq  1995]  where  only  part  of  the  shortest  path  array  is  cached,  in  a  manner  that  per¬ 
mits  the  uncached  distances  to  be  recovered  in  time  at  most  0(|F|). 

To  modify  the  main  algorithm  in  Figure  1  with  improved  forward  checking,  we  only  need 
do  two  things.  First,  we  replace  the  forward-checking  routine  with  one  that  uses  the  FC- 
condition,  as  shown  in  Figure  2.  Second,  we  add  one  Hne  to  the  main  program  (Basic-DTP); 
specifically, 

i’ — maintain-consistencyfir^  S) 

should  be  inserted  between  Hnes  4  and  5.  Each  time  a  new  variable  is  assigned  a  value  {C.  c}, 
the  constraint  is  propagated  in  S  by  maintain-consistency(^r,  5),  which  can  be  implemented 
with  either  fuU  path  consistency  or  with  directional  path  consistency,  as  described  above. 


4  This  theorem  was  suggested,  but  not  proved,  in  [Oddi  and  Cesta  2000]  see  Appendix  A  for  its  proof. 
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The  comparison  of  overall  complexity  in  each  node  does  not  yield  an  obvious  'Test”  ap¬ 
proach.  As  already  noted,  (i)  the  basic  forward-check  procedure  requires  0(v|F|^)  time  for 
each  node  in  the  CSP  search  tree;  (ii)  computing  full  path  consistency  and  checking  the  FC- 
condition  requires  0(|F|^  +  ^);  and  (iii)  computing  directional  path  consistency  for  FC- 
checking  requires  0(|F|  v  |F|  ).  In  addition,  since  assignment^  is  built  incrementally  by  add¬ 
ing  constraints  on  each  new  node,  we  can  use  incremental  versions  of  these  previous  tech¬ 
niques  to  build  S,  namely  incremental full path  consistent  (IFPC)  [Mohr  and  Henderson  1986]  and 
incremental  directional  path  consistent  (IDPC)  [Chleq  1995].  The  incremental  versions  drop  the 
exponent  in  aU  the  above  complexities  to  quadratic  and  so  (i)  takes  time  0(v|F|^),  (ii)  takes 
time  0(|F|^  +  V)  and  (iii)  0(|F|^+ v|F|).  Thus,  the  worst-case  comparison  favors  maintaining 
fuU  path  consistency  (i.e.  the  distance  array)^.  The  average  case  comparison  cannot  easily  be 
resolved  theoretically  and  further  experiments  are  required  to  determine  under  which  condi¬ 
tions  each  method  is  the  best.  In  our  experiments,  reported  below  in  Section  6  we  used 
method  (ii),  maintaining  the  distance  array  at  every  node. 

3.  Previous  Pruning  Techniques  for  DTP  Solving 

Once  the  DTP  problem  has  been  cast  as  one  of  solving  a  meta-CSP,  a  number  of  different 
backtracking  search  techniques  can  be  used  to  increase  efficiency  by  early  pruning  of  dead-end 
branches.  The  idea  in  pruning  techniques  is  to  utilize  the  underlying  semantics  of  the  values  of 
the  meta-CSP,  namely  the  fact  that  they  express  constraints  on  some  STP,  to  make  inferences 
regarding  the  infeasibiHty  of  certain  regions  of  the  search.  In  this  section  we  describe  three 
methods  previously  considered  in  the  literature:  Conflict-Directed  Backjumping  (CDB) 
(used  in  [Stergiou  and  Koubarakis  2000]  Semantic  Branching  (SB)  (used  in  [Oddi  and  Cesta 
2000]  and  [Armando,  CasteUini  et  al.  1999]),  and  Removal  of  Subsumed  Variables  (RSV) 
(used  in  [Armando,  CasteUini  et  al.  1999]).  In  the  next  section  we  wiU  add  No-good  Re¬ 
cording  (NR)  (also  caUed  no-good  learning),  and  hence  throughout  both  these  sections  we 
wiU  be  particularly  concerned  with  the  theoretical  and  technical  issues  that  arise  in  successfuUy 
integrating  these  pruning  strategies  with  one  another  and  with  no-good  recording. 

3.1  Conflict-Directed  Backjumping  for  DTPs 

The  simplest  algorithms  for  solving  CSP  problems  rely  on  chronological  backtracking,  in 
which  the  failure  of  a  partial  assignment  of  values  to  variables  results  in  backtracking  to  the 
point  in  the  search  just  before  the  most  recent  assignment  of  a  value  to  a  variable  was  made. 
Previous  work  has  shown  that  backtracking  can  be  made  more  efficient  by  instead  restarting 
the  search  at  a  more  carefuUy  selected  point:  techniques  developed  for  this  include  Dynamic 
Backtracking  [Ginsberg  1993],  and  Conflict  Directed  Backjumping  {CDB)  [Prosser  1993; 
Chen  and  Beek  2001].  In  these  approaches,  when  a  dead  end  is  encountered,  the  search  back¬ 
tracks  to  the  most  recently  assigned  variable  that  is  related  to  the  failure.  The  variables  that  are 
unrelated  to  the  failure  are  backjumped  over,  since  trying  to  assign  different  values  for  them 
wiU  result  in  the  same  dead  end. 


5  This  is  because  the  worst-case  bounds  are  tight. 
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(a) 


(b) 

Figure  3:  The  chronological  backtracking  algorithm  on  a  DTP. 


It  is  obvious  that  to  implement  CDB,  it  is  necessary  to  be  able  to  identify  the  culprit  of  the 
failures,  i.e.,  the  variables  that  participate  in  the  constraints  that  lead  to  failure.  Stergiou  and 
Koubarakis  [Stergiou  and  Koubarakis  2000]  present  a  method  for  culprit  identification  in  DTP 
solving,  which  they  caU  the  dependent  pointers  scheme.  This  scheme  is  based  on  the  fact  that,  dur¬ 
ing  DTP  solving,  whenever  an  assignment  is  extended  to  A’— A.  {C.  c],  forward  checking 
is  performed.  If  a  value  c’  is  removed  from  some  domain,  then  the  most  recent  value  assign¬ 
ment,  {C.  c},  must  directly  contribute  to  its  removal.  In  the  approach  of  Stergiou  and  Kouba¬ 
rakis,  a  dependency  pointer  from  to  c  is  stored.  If  the  algorithm  subsequently  needs  to  back¬ 
track  because  the  domain  of  some  variable  has  been  reduced  to  — i ,  the  algorithm  checks  the 
dependency  pointers  for  values  that  were  removed  from  that  variable’s  domain,  and  follows 
the  one  that  points  to  the  most  recently  assigned  variable,  thereby  backjumping  over  any  ir¬ 
relevant  variables. 

Although  the  dependency  pointer  scheme  achieves  CDB,  it  does  not  integrate  weU  with 
semantic  branching  and  no-good  recording.  We  thus  developed  an  alternative  scheme  for  cal¬ 
culating  the  culprit  of  a  failure.  Our  technique  returns  the  variables  of  the  current  assignment 
that  are  involved  in  the  failure;  these  can  be  used  in  a  manner  similar  to  dependency  pointers, 
to  backjump  to  the  most  recent  relevant  variable.  Additionally,  however,  the  returned  set  of 
variables  can  be  used  as  justifications  in  no-good  recording,  as  described  in  Section  4. 
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justification-value(c :  y-x  =  b  ,S) 

\.p  =  shotest-path( y,  x,  S) 

2.  Return  vars(p  .  c} 

Figure  4:  Function  justification-value 

Our  approach  is  straightforward,  and  builds  directly  on  the  fact  that  backtracking  is  re¬ 
quired  only  when  forward-check  has  reduced  the  domain  of  some  variable  to  the  empty  set 
by  removing  every  value  ^  of  that  domain.  This  in  turn  implies  that  every  c-  that  was  in  the  do¬ 
main  is  part  of  a  negative  cycle p-  formed  by  constraints  q  ,  We  introduce  the  technique 

with  an  example. 

Example  1:  Figure  3  illustrates  the  processing  of  the  following  DTP: 

C,  :  {Cjj  :j-x=5} 

Q:  =  .  {C22  :  x -j  ^  -10}  . 

Q  :  :  V  —  X  =  S}  .  {c^2  'Z~  ^0} 

Q-.  5}  .  {c^2--J-n>^-10} 

Q  :  {^51  ■■y-K.=  -'^0) 

In  the  figure,  the  top  two  boxes  (a)  represent  a  ''snapshot”  of  the  DTP  solving  process:  the 
left-hand  side  shows  the  meta-CSP  search  tree,  and  the  right-hand  side  shows  the  STP  entailed 
by  the  current  assignment.  The  bottom  two  boxes  (b)  show  a  snapshot  later  in  the  process.  At 
the  time  of  Figure  3(a),  assignments  have  been  made  to  C2^  and  Q.  The  assignments  cho¬ 
sen  are  indicated  by  the  gray  ovals  while  the  white  ovals  indicate  already  explored  nodes.  Note 
that  values  that  have  been  ruled  out  by  forward-checking  are  crossed  out  in  the  STP  diagrams. 
For  instance,  the  assignment  of  (y  —  x  =  5)  to  Q  rules  out  the  possibility  of  assigning  C22  (x  — 
j=  -10)  to  Q;,  and  so  this  value  is  crossed  out  in  the  right-hand  part  of  Figure  3(a). 

Figure  3(b)  shows  a  later  point  in  the  processing,  by  which  an  assignment  has  also  been 
made  to  Q  (specifically  Q .  c^)).  At  this  point,  forward-checking  wiU  eliminate  both  possible 

values  for  Q ,  because  they  participate  in  negative  cycles.  These  cycles  are  independent  of  the 
assignment  made  to  Q ,  which  should  thus  be  backjumped  over.  That  is,  and  c^2 
moved  from  d(Cp  because  they  form  the  negative  cycles  (in  the  STP): p^  —  {  C21 ,  c^))  and p2 

—  (q; ,  C2-1  ^  c^p  The  variables  that  participate  in  the  failure  then  are  vars(^;j  .  vars(^2J  ~  {Q 
,  Q  }  .  {C^ ,  C2  ,  Q  }  =  {C; ,  Q  ,  Q  },  where  varsp)  are  the  variables  whose  value 

assignments  are  the  constraints  in p. 

It  is  apparent  that  our  technique  requires  the  identification  of  a  negative  cycle  for  each  re¬ 
moved  value  by  forward  check.  This  can  be  implemented  by  maintaining  a  predecessor  array^ 
[Cormen,  Leiserson  et  al.  1990]  when  calculating  the  shortest  path  array.  Entry  <ij>  of  the 
predecessor  array  contains  nil  when  otherwise  it  is  a  predecessor  of  j  on  the  shortest  path 
from  /.  It  should  be  updated  by  the  function  maintain-consistency,  which  can  be  done  with¬ 
out  changing  the  time  complexity  of  the  function.  When  a  value  c  \j  —  x—  b  completes  a  nega¬ 
tive  cycle  (i.e.  the  FC-condition  holds),  we  foUow  the  predecessor  array  to  retrieve  the  shortest 
path p  fromj/  to  x  and  return  vars(p  .  (x,  j)),  where  {x,j)  is  the  edge  from  x  to j.  The  pseudo- 


^  Recall  that  the  predecessor  array  stores  a  predecessor  of  j  on  the  shortest  path  from  i  in  all  entries  <y>  that  are  not  on  the 
main  diagonal. 
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Figure  5:  Example  DTP  for  removal  of  subsumed  variables  and  semantic  branching. 


code  for  implementing  this  approach  is  given  in  Figure  4:  the  function  justification-value  re¬ 
turns  the  justification  (i.e.  the  culprit  set  of  variables)  for  the  removal  of  value  c  :j  —  x—b  from 
the  domain  of  its  variable,  given  an  STP  S  that  corresponds  to  the  current  assignment. 

3.2  Removal  of  Subsumed  Variables 

The  main  idea  of  the  heuristic  that  we  will  call  Removal  of  Subsumed  Variables  (RSV)  is 
that  if  a  disjunct  of  a  variable  C-  is  already  satisfied  by  the  current  assignment  A.,  there  is  no 
reason  to  try  other  values  in  the  variable’s  domain  under  assignment  because  either  (i)  the 
current  assignment  leads  to  a  solution,  and  since  is  already  satisfied  under  A,  C-  is  satisfied  in 
the  solution,  or  (ii)  there  are  no  solutions  under  ^  and  trying  other  values  for  C-  will  only  re¬ 
strict  ^  even  further,  with  no  possibility  of  discovering  a  solution.  We  now  proceed  by 
formalizing  this  idea^. 

Definition  2:  A  value  is  subsumed  by  an  STP  S  (equivalently  by  an  assignment 
A  that  implies  T)  if  and  only  if  the  constraint  c-  always  holds  in  any  exact  solution  of  S. 
(Recall  that  an  exact  solution  to  a  DTP  D  is  an  assignment  of  numbers  to  the  time 
points  in  D.)  A  variable  is  subsumed  by  an  STP  S  if  and  only  if  there  is  a  value  c-j 
in  the  domain  of  C-  that  is  subsumed  by  S. 

Theorem  2:  A  value  c-j  \  j  —  x  —  b  is  subsumed  by  an  STP  S  if  and  only  if  d^(S)  —  b 
{Subsumption-Conditioti)^  where  d^j(S)  is  the  distance  between  x  andj/  in  S. 

Theorem  3:  Let  D—<Vy  C>  be  a  DTP,  let xl  be  an  assignment  on  D  (i.e.  a  compo¬ 
nent  STP),  and  let  C-  be  a  variable  subsumed  by  A.  Then  xl  is  a  solution  of  D  if  and 
only  if  it  is  a  solution  oi  D’—<Vy  C  —  CA- 

Corollary  1:  Let  xl  be  a  partial  assignment  during  a  DTP  search,  U  be  the  unassigned 
variables,  and  Sub  be  the  set  of  subsumed  variables  in  U.  If  xl  can  be  extended  to  a  so¬ 
lution  over  variables  in  U  —  Sub,  it  can  be  extended  to  a  solution  over  variables  in  U. 

In  other  words,  we  can  remove  the  subsumed  variables  from  the  unassigned  variables 
during  search.  The  solution  to  the  reduced  problem  is  a  solution  to  the  original  one. 

The  proofs  for  the  above  theorems  and  corollaries  are  in  Appendix  A. 


^  RSV  was  first  used  by  [Oddi  and  Cesta  2000]  but  without  providing  a  proof. 
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Figure  6:  The  search  tree  showing  the  effects  of  the  removal  of  subsumed  variables. 


Example  2:  The  ramifications  of  the  above  corollary  are  shown  pictorially  Figure  6, 
which  depicts  a  search  without  the  use  of  RSV  for  a  solution  to  the  DTP  in  Figure  5. 
The  variables  are  considered  in  order  (1-6).  Initially,  —  {Q  .  c^^}.  This  is  then  ex¬ 
tended  to  A2  —  {C;  .  ,  Q  .  62^.  Without  removing  the  subsumed  variables,  the 

next  assignment  would  be  ,  Q  .  C2-1  ,  Q  .  Notice  though  that 

value  ,  and  thus  variable  Q  is  subsumed  by  because  together  —  x  =  5  and 
.•  X  —  ^  —  5  imply  that  j-x— which  subsumes  the  constraint  'J~Z~  Thus,  by 
Corollary  1  Q  can  safely  be  removed  from  the  search  underneath  the  subtree  of 
Suppose,  however,  that  it  is  not  removed.  Then  the  search  will  proceed  as  in  Figure  6. 
When  the  search  of  subtree  T;  in  figure  fails,  as  it  will  in  this  particular  example,  the 
search  continues  by  trying  the  other  value  of  Q  and  so  =  {Q  .  c^f  ,  C2  .  C21  ,  C3 

By  Corollary  1,  since  A^  has  failed,  A^  wiU  fail  too.  By  removing  the  subsumed 
variable  Q  in  this  particular  example,  subtree  and  the  node  corresponding  to  A^  in 
the  figure  would  have  been  safely  pruned.^ 

3.3  Semantic  Branching 

A  third  pruning  method  used  in  solving  DTPs  is  semantic  branching  iSB)^  which  has  been 
shown  to  be  very  effective  [Armando,  CastelHni  et  al.  1999].  Like  RSA,  SB  reHes  on  the  se¬ 
mantics  of  the  constraints  in  the  DTP,  i.e.,  on  the  fact  that  they  encode  numeric  inequalities. 
The  basic  idea  of  semantic  branching  is  the  following.  Suppose  that  during  search  the  as¬ 
signment  A  .  {C-  .  is  expanded  in  every  possible  way  but  it  leads  to  no  solution.  That 

means  that  in  any  solution  that  is  an  extension  of  A,  if  there  is  any,  the  constraint  c-j  does  not 
hold.  Thus,  the  negation  of  this  constraint  has  to  hold  in  any  such  solution.  In  other  words,  if 
c-j  is  the  constraint  x  —y  =  b  and  we  know  c-j  does  not  hold,  then  in  any  solution  that  is  an  ex¬ 
tension  of  assignment  xl,  has  to  be  true,  i.e.  it  must  be  the  case  thatj  —  x  <  -b.  Thus,  when 


8  The  node  corresponding  to  XI3  would  also  have  been  pruned. 
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search  ''branches”  after  failing  to  extend  A  .  {C-  .  to  a  solution,  and  tries  a  different 

value  for  C-  for  the  rest  of  the  search  under  A,  we  can  assume  holds.^  The  constraint 
often  tightens  the  STP  that  corresponds  to  the  current  assignment  A',  explicitly  adding  this 
constraint  can  lead  to  values  in  other  variable  domains  being  removed  earlier  than  they  other¬ 
wise  would  have  been. 

Notice  that  with  SB  the  current  assignment  ^  at  any  point  in  processing  no  longer  stands 
in  a  one-to-one  correspondence  with  an  STP  S.  Instead  S  is  the  union  of  the  values  assigned  to 
variables  in  and  aU  the  current  semantic  branching  constraints. 


Example  3:  To  see  how  SB  prunes  the  search  space,  we  compare  the  search  space  for 
the  DTP  of  Figure  5  without  and  then  with  semantic  branching.  Suppose  the  algo¬ 
rithm  has  already  assigned  A-,  —  {Q  .  c-i-,  ,  Q  .  C2-1  ,  Q  .  as  shown  in  Figure  8. 

(On  the  left  is  the  search  of  the  meta-CSP  and  on  the  right  is  the  implied  STP.)  The  x- 


Figure  7:  Semantic  Branching  Example  (c) 


crossed  edges  are  the  ones  removed  by  forward  checking  while  the  fiUed  nodes  are  the 
ones  that  belong  to  the  current  assignment.  In  Figure  9  the  assignment  is  extended  to 
A2  —  {C;  .  ^  C2  •  ^21  ?  ^3  •  ^31  y  Q.  This  assignment  fails  because  both  val¬ 

ues  in  D(C^)  are  removed. 


9  The  implementation  of  semantic  branchiag  is  discussed  iu  Sec.  6.6. 
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The  search  then  continues  by  trying  a  different  value  for  Q  (Figure  7  ).  Finally,  the 
search  reaches  a  dead  end  again  because  both  values  in  D(CJ  are  removed  (Figure  10), 
after  which  it  will  backtracks  back  to  node  Q  and  continue  the  search. 

Had  we  used  semantic  branching  however,  when  we  branched  to  try  the  second 
value  of  C4  we  would  have  explicitly  added  the  constraint  <-e41,  as  shown  in  boldface 
in  Figure  11.  The  constraint  <-e41  allows  forward  checking  to  eliminate  value  c61  im¬ 
mediately,  thus  reaching  a  dead  end.  In  this  simple  example,  SB  prunes  only  one  node, 
the  one  that  assigns  C5  .  c51  (last  node  in  left  picture  of  Figure  10),  but  in  general 

SB  can  prune  an  arbitrarily  large  number  of  nodes. 

As  is  noted  in  [Oddi  and  Cesta  2000],  Semantic  Branching  is  only  useful  when  the  disjuncts 
in  each  constraint  are  not  mutually  exclusive.  For  example,  in  scheduling  applications  where 


the  constraints  are  typically  of  the  form  {A  <  B  or  B  <  A},  when  the  first  disjunct  fails,  SB 
wiU  add  its  negation  A  >  B,  having  no  pruning  effect,  since  the  next  disjunct  B  <Ais  the  same 
constraint. 
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Figure  11:  Semantic  Branching  example  (e) 


4.  No-good  Recording 

No-good  recording  (also  called  no-good  learning)  is  a  powerful  pruning  technique  for  solv¬ 
ing  general  CSPs  [Dechter  1990;  Frost  and  Dechter  1994;  Ginsberg  and  McAUester  1994; 
Schiex  and  VerfaiUie  1994;  Schiex  and  VerfaiUie  1994;  Yokoo  1994;  Dechter  and  Frost  1999] 
and  SAT  problems  [Roberto  J.  Bayardo  and  Schrag  1977].  In  this  section,  we  adapt  this  tech¬ 
nique  to  DTP  solving.  Intuitively,  a  no-good  is  an  assignment  of  the  variables  that  cannot  lead 
to  a  solution,  and  is  thus  either  an  induced  or  explicit  constraint  of  the  CSP.  It  is  important  not 
to  confuse  no-goods  with  semantic  branching  constraints.  No-goods  are  constraints  of  the 
meta-CSP,  while  SB  constraints  are  constraints  of  the  component  STP  associated  with  one 
particular  (possibly  partial)  assignment  to  the  variables  of  the  meta-CSP. 

In  our  Epilitis  algorithm  we  use  no-goods  for  two  purposes:  (i)  for  pruning  the  search 
space  and  (ii)  as  heuristic  information  to  estimate  which  variables  constrain  the  remaining 
search  space  the  most.  This  section  deals  with  the  former,  while  Section  6.4  with  the  later. 

We  begin  by  defining  no-goods  in  general,  for  an  arbitrary  CSP  C>.  In  our  definitions, 
we  wiU  use  .  C  to  denote  the  constraints  in  C  that  involve  only  the  variables  in  X,  where  X 
.  K 

Definition  3:  A  no-good  of  CSP  <1Y,  C>  is  a  pair  />,  where  X  is  a  set  of  for¬ 
bidden  assignments  to  a  subset  of  X,  and  /,  called  the  no-good  justification  or  culprit, 
is  a  subset  of  X  such  that  no  solution  of  the  CSP  <X,  C)  >,  given  the  specified  do¬ 
mains  for  the  variables  in  X,  contains  the  assignments  in  A. 

Example  4:  Consider  a  CSP  where  V  —  {a,  by  c) ,  D(a)  —  T>{B)  =  D(<r)  =  {1,2},  with 
the  following  constraints:  C  —  {C^  —{<r-{a  .  1  .  b  .  2)},  Q  —{<r-{a  .  2  .  c  .  2)},  Q 
=  {<-(/?.  2.  c.  2)},  Q  ={<-(^^  .  1.  c.  1)  },  Q  ={<-(^^  .  1.  c.  2) }}.  Each  con¬ 
straint  C- trivially  induces  a  no-good.  For  example,  C;  implies  that  <{  a  .  b  .  2], 

{ay  ^}>  is  a  no-good.  Now  notice  that  if  an  assignment  were  to  include  a  .  /,  con¬ 

straint  Q  would  preclude  c  from  taking  value  1  and  Q  would  preclude  c  from  taking 
value  2.  Since  these  are  the  only  values  in  the  original  domain  of  q  we  can  infer  the 
new  constraint  {<r-(a  .  /)}.  Thus,  the  pair  <Ay  />,  where  A={a  .  /}  is  also  a  no- 
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good,  for  some  justification  J.  What  is  the  justification  J?  The  constraints  that  imply 
the  no-good  are  Q  and  Q:  it  is  as  a  result  of  these  two  constraints  that  we  cannot  as¬ 
sign  a  the  value  /.  Thus,  the  variables  that  ''justify”  the  no-good  are  the  variables  of 
these  two  constraints  and  so  /  =  {a^  c}.  Then  Cj=  {Q ^  Q}  and,  as  the  definition 
requires,  yl={a  .  /} cannot  be  part  of  any  solution  to  the  CSP  <1/,  Notice  that 

a  no-good  J>  does  not  only  depend  on  the  constraints  C  of  the  CSP,  but  also  on 
the  domains  of  the  variables.  If  the  domain  of  c  in  this  example  contained  more  values 
than  1  and  2  we  could  not  have  inferred  that^={^^  .  /}  is  an  induced  constraint. 

The  above  example  illustrates  a  particular  point:  knowing  a  set  of  no-goods,  we  may  be  able 
to  infer  other  no-goods.  The  following  two  theorems  present  two  methods  for  such  infer¬ 
ences. 

Theorem  4:  Let  <A,J>  be  a  no-good.  Then  <A.  /,  J>  is  also  a  no-good,  where  ^  . 

V  denotes  the  assignment  that  results  from  projecting  assignment  ^  on  the  variables 
of  V  (Theorem  3.2  in  [Schiex  and  Verfaillie  1994]). 

Intuitively,  the  theorem  states  that  we  can  reduce  the  assignment  of  a  no-good,  by  only 
considering  the  variables  in  the  justification.  For  example,  <  {a  .  1y  c  .  2},  {a,  ^}>  is  a  no¬ 
good,  then  <{^^  .  1,c.  2).  {a y  b],  {ay  b]>  —  <{a  .  is  also  a  no-good. 

Theorem  5:  Let^  be  a  (partial)  assignment  of  the  variables  in  17,  ^^be  an  unassigned 
variable  in  17,  and  be  all  the  possible  extensions  of  ^  along  v^y  using 

every  possible  value  of  D(^^)  .  If  <A^  y  J^>  ,  . . .,  <A^ ,  /^>  are  no-goods,  then  <A  ,  .  • 

J>  is  a  no-good  (Corollary  3.1  in  [Schiex  and  Verfaillie  1994]). 

Example  5:  Theorem  5  is  exactly  what  we  used  intuitively  in  Example  4  to  infer  that 
<{a  .  1}y  [ay  c}>  is  2i  no-good.  Let  us  illustrate  now  the  same  CSP  and  the  same  deri¬ 
vation  again  in  light  of  Theorem  5.  If  we  let  ^  .  /},  we  see  that  Af  ={a  .  1  y  c 

.  /}  and  —  {^  •  1  y  c  .  2]  are  all  the  possible  extensions  of  ^  along  variable  c. 

Trivially  (see  the  discussion  in  Example  A)y  <  [a  .  1y  c  .  /}  ,  [uy  c]>  and  <{a  .  1y  c 

.  2]  ,  {ay  c]>  are  no-goods,  or  equivalently  <A^y  {ay  c}>  and  <A2  ,  {ay  c}>  are  no¬ 
goods.  By  the  theorem  we  infer  that  <Ay  {ay  4^  is  a  no-good  too. 

4.1  Building,  Recording,  and  Using  No-Goods  during  Search 

Suppose  we  design  our  search  algorithm  so  that,  given  a  partial  assignment^,  it  explores  aU 
extensions  of  and  always  returns  one  of  two  results:  a  solution,  or  a  justification  J  for  the 


10  The  reader  might  wonder  why  we  define  a  no-good  as  the  pair  J>,  where  the  justification  J  is  the  set  of  the  variables 
involved  in  the  constraints  that  imply  vt,  instead  of  having  J  to  be  the  set  of  the  actual  constraints.  Indeed,  Schiex  and 
Verfaille  [Schiex  and  Verfaillie  1994]  record  the  involved  constraints  as  the  no-good  justifications.  For  our  current  exam¬ 
ple,  the  no-good  would  be  <{<^  .  1},  {C4 ,  C^}>  instead  of  <{<^  .  1},  {a,  c}>.  Notice  that  Cj  =  C[a,  b)  is  a  superset  of  {C4  , 

C5}.  In  general.  Definition  3  leads  to  less  specific  justifications  than  those  discovered  using  the  Schiex  and  Verfaillie 
method.  However,  when  employing  no-goods  for  solving  the  Disjunctive  Temporal  Problem,  it  is  more  convenient  to 
encode  and  use  as  justifications  sets  of  variables  than  sets  of  constraints,  especially  since  in  the  DTP  the  constraints  are 
implicit).  The  two  definitions  of  no-goods  are  equivalent  for  all  purposes  of  this  paper.  For  a  more  thorough  discussion 
on  the  subject  see  [Richards  1998]. 
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failure  of  all  the  extensions  of  A.  In  other  words,  we  assume  that  invoking  a  search  on  the  suc¬ 
cessor  A.  .  {v  .  u-i)  returns  either  a  solution  or  the  no-good  <A  .  {v  .  /;>.  By 

Theorem  5,  if  aU  successors  of  ^  fail  returning  <A.  {v  .  then  we  can  infer  the  new 

no-good  <A,  .  /^>,  which  can  be  further  reduced  to  the  no-good  <A..  .  /^>  by  Theorem 

4.  This  no-good  has  a  smaller  forbidding  assignment  than  aU  the  no-goods  of  the  successors 
and  it  can  be  returned  recursively  to  the  parent  of  the  current  node  to  explain  why  ^  failed  to 
be  extended  to  a  solution.  Thus,  if  the  leaves  of  the  search  return  a  no-good  with  a  justification 
for  the  failure,  the  internal  nodes  can  infer  and  build  smaller  no-goods  using  the  method  just 
described. 

The  preceding  discussion  shows  how  to  propagate  constraints  from  the  leaf  nodes  through 
internal  nodes  of  the  CSP.  The  remaining  question  is  how  to  generate  the  no-goods  at  the 
leaves.  Building  no-goods  at  the  leaves  is  easy  for  standard  CSPs:  if  the  current  assignment  at 
the  leaf  violates  a  constraint  C,  then  the  no-good  <Aj  l/^>  is  returned,  where  contains  the 
variables  in  V  that  appear  in  the  constraints  in  C.  If  more  than  one  constraint  C  is  violated,  we 
can  arbitrarily  select  one  to  return^b 

In  the  meta-CSP  of  a  DTP,  the  constraints  among  the  CSP  variables  are  impHcit  and  have 
to  be  inferred,  and  so  it  is  not  as  straightforward  to  determine  what  justification  to  return  when 
a  constraint  is  violated.  We  distinguish  two  cases  for  when  assignment  ^  violates  a  constraint 
and  correspondingly  two  ways  to  form  a  justification  for  the  failure: 

1.  A  is  a  superset  of  A^ for  some  already  recorded  no-good  <A)  J>.  In  this  case  J  is  returned  as  the 
justification. 

2.  vT  corresponds  to  an  STP  that  is  inconsistent.  Suppose  that  p  is  the  negative  cycle  in  the  in¬ 

consistent  STP.  If  there  are  no  semantic  branching  constraints  added,  then  this  negative 
cycle  is  formed  entirely  from  value-variable  assignments  in  A.  If  vars(p)  are  the  variables 
whose  value  assignments  are  the  STP  constraints  in  p^  the  set  vars(p)  is  the  justification 
that  should  be  returned  .  For  example,  if  assignment  ^={C;  .  ,  Q  .  C/  ?  ^3  • 

^y;}and p—{c^p  C2fy  then  the  justification  J  =  {Cp  Q}  should  be  returned.  However,  if 
semantic  branching  constraints  are  added,  then  they  might  also  participate  in  the  nega¬ 
tive  cycle py  e.g.  if  assignment  H={C;  .  ,  Q  .  ,  Q  .  c^^  }and p—{c^p  C2-1  y  v)y 

where  ^  is  a  semantic  branching  constraint.  In  this  case,  the  set  of  variables  that  consti¬ 
tutes  the  culprit  of  the  failure  is  vars(p)  and  aU  the  variables  that  justify  the  addition  of  v. 
Assuming  that  we  have  a  way  of  obtaining  the  justification  of  the  semantic  branching 
constraints,  denoted  by  the  function  just(v)  for  a  constraint  the  justification  to  be  re¬ 
turned  should  be  is  vars(p)  .  Jusfvf  where  v-  are  aU  the  semantic  branching  constraints 
that  participate  in  the  negative  cycle  p.  In  order  to  implement  function  just(v)  we  need  to 
store  the  pairs  <Vy  J>  where  ^  is  a  semantic  branching  constraint  that  holds  in  the  cur¬ 
rent  assignment  and  J  the  justification  of  the  most  recent  failure  prior  to  the  addition  of 
r  (i.e.  the  failure  that  led  to  the  addition  of  r). 


In  [Schiex  and  Verfaillie  1994]  the  idea  of  returning  more  than  one  justification  per  failure  is  explored,  but  this  is  outside  the 
scope  of  this  paper. 
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Notice  that  case  (1)  requires  that  during  search  the  current  assignment  ^  is  checked 
against  all  recorded  no-goods  to  determine  whether^  .  for  some  no-good  J>.  This 
lookup  operation  imposes  a  significant  overhead  for  using  the  recorded  no-goods  (see 
[Tsamardinos  2001]  for  an  efficient  implementation  of  no-good  lookup  scheme).  Recording 
more  no-goods  provides  better  chances  for  pruning  the  search  space;  however,  it  increases  the 
time  for  the  lookup  operation.  Thus,  one  needs  to  determine  which  no-goods  to  keep  among 
all  possible  no-goods  discovered  during  search.  The  easiest  scheme  is  to  Hmit  the  size  of  the 
no-goods  recorded  by  a  fixed  constant  k:  a  no-good  assignment  is  recorded  only  if  it  contains 
less  than  k  value-variable  assignments  (independent  of  the  size  of  the  justification  set).  In  the 
experiments  we  conducted  we  determined  the  best  value  for  k  for  the  range  of  problems  we 
tested,  as  described  in  Section  6.4. 

5.  Integrating  all  Pruning  Methods:  The  Epilitis  Algorithm 

We  are  now  ready  to  describe  our  algorithm  for  DTP  solving.  Called  Epilitis,  the  algorithm 
combines  aU  the  pruning  methods  used  in  the  previous  literature  on  DTP  solving — namely 
Conflict  Directed  Backjumping,  Removal  of  Subsumed  Variables,  and  Semantic  Branching — 
and  it  adds  in  the  no-good  recording  scheme  of  discussed  in  Section  4.  The  main  difficulty  in 
designing  the  algorithm  is  that  no-good  recording,  CDB  and  SB  interact  and  special  attention 
is  required  to  combine  them.  Here  we  present  a  high-level  description  of  the  algorithm  shown 
in  Figure  12;  complete  details,  sufficient  for  implementation,  are  provided  in  the  Appendix  A. 

As  in  the  previous  approaches,  EpiHtis  attacks  a  DTP  by  attempting  to  solve  the  associated 
meta-CSP,  searching  for  a  consistent  component  STP.  It  takes  three  arguments: 

H,  the  current  assignment  of  values  (of  the  form  xj  <—B)  to  variables 
U,  the  yet-to-be-assigned  variables 

S,  the  current  induced  STP,  which  is  represented  by  a  distance  array,  a  precedence  ar¬ 
ray,  and  a  set  of  pairs  />,  such  that  p  are  the  semantic  branching  constraints  justified 
by  the  meta-level  constraints  involving  the  variables  in  J. 

When  EpiHtis  is  initiaUy  invoked  to  solve  a  DTP  A  —  —i^  U  —  C,  the  variable  do¬ 

mains  are  initiaHzed  as  in  the  basic  DTP  algorithm  of  Figure  1,  and  the  distance  and  predeces¬ 
sor  arrays  are  empty,  as  are  the  SB  constraints.^^ 


Recall  that  the  distance  army  is  the  all-pairs  shortest  path  matrix,  and  the  predecessor  array  stores  a  predecessor  of  j  on  the 
shortest  path  from  i  in  all  entries  <y>  that  are  not  on  the  main  diagonal. 
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Epilitis(^,  U,  S) 

1.  /*  Removal  of  Subsumed  Variables)  */ 

2.  For  all  variables  x  in  U, 

3.  Remove  x  from  U  if  for  any  value  v  in  d(x)  the  Subsumption  Condition  holds  in  the 

4.  current  STP  S. 

5.  EndFor 

6.  If  U=^  then 

7.  Stop  and  report  ^  as  a  solution 

8.  Else 

9.  Select  a  variable  x  'mU 

10.  For  all  values  v  in  the  current  domain  of  x,  d(x) 

1 1 .  forward-check  v 

12.  If  forward-check  fails  with  justification  Just, 

13.  record  <^.  {x.  v} .  JwV,  (No-good  recording) 

14.  Else, 

15.  Try  extending  A  by  {x.  v}  (Recursively  call  Epilitis). 

16.  If  the  call  returns  with  justification  Ji  that  does  not  involve  x, 

17.  backjump  and  return  Jf  (Conflict-Directed  Backjumping) 

18.  Endlf 

19.  Endlf 

20.  If  value  v  fails,  add  reverse(v)  to  S,  Endlf  (Semantic-Branching) 

21.  EndFor 

22.  If  all  values  v  (in  the  original  domain  of  x,  D(x))  have  failed  or  been  removed  from  D(x) 

23 .  with  justifications  Ji 

24.  record  <v4..  [Jf,.  iJ;>,  return  .  (No-good  recording) 

25.  Endlf 

26.  Endlf 


Figure  12:  High-level  description  of  Epilitis  algorithm. 


On  any  (recursive)  call,  if  U=—!  then  ^  represents  a  solution  to  the  DTP.  If  any  variable  in 
U  is  subsumed  by  S,  then  it  is  removed  from  U.  Next,  a  variable  x  in  U  is  selected  and  an  at¬ 
tempt  is  made  to  extend  xl  by  making  an  assignment  to  x.  Otherwise,  each  value  Vj^  in  d(x)  is 
considered  in  turn  and  xl  is  extended  to  A'  —A  .  {x  .  v^}y  while  constraint  Vj^  is  propagated 
in  S.  If  forward-checking  a  value  Vj^  reduces  the  domain  of  some  variable  to  the  empty  set,  then 
a  dead-end  has  been  reached.  At  this  point,  Epihtis  records  the  no-good  <A.  {x.  v}.  Just^ 
Just>  where  Just  is  the  justification  for  the  failure  as  discussed  in  4.1.  Otherwise,  if  forward¬ 
checking  does  not  lead  to  a  dead-end,  then  Epihtis  is  recursively  invoked. 

If  a  dead-end  has  been  reached  for  every  possible  extension  of  x  among  ah  Vj^  then  we  build 
and  record  another  no-good  <A,.  .  JA  where  are  the  justifications  for  eachxl  .  {x 

.  yJfaiUng. 

CDB  is  implemented  with  the  following  scheme:  If  while  recursively  calling  Epihtis  with 
assignment  =xl  .  {x  .  a  failure  occurs  with  justification  /,  then  there  is  no  need  to  try 
another  value  Vp  if  x  does  not  appear  in  J:  if  x  is  not  in  the  culprit  of  the  failure,  the  same  dead¬ 
end  whl  be  encountered  again  for  xl  .  {x  .  v^.  Thus,  we  can  stop  trying  any  remaining  val¬ 
ues  in  the  domain  of  x,  and  backjump  over  x. 

Finahy,  SB  is  implemented  by  propagating  the  reverse  in  the  current  STP  S,  when 

xl  .  {x  .  Vj^}  leads  to  fahure.  However,  recah  that  in  order  to  create  the  correct  justifications 
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for  building  no-goods  and  performing  CDB  the  pairs  <iv^  J>  of  the  current  set  of  semantic 
branching  constraints  need  to  be  maintained,  where  J  is  the  justification  for  adding  m 

6.  Experimental  Results 

6.J  Experimental  Setup 

We  next  describe  the  results  from  a  series  of  experiments  that  we  ran  on  Epilitis  and  the 
solver  of  Armando,  Castellini,  and  Giunchiglia  called  TSAT,  publicly  available  at 
http://www.mrg.dist.unige.it/~drwho/Tsat.  (As  described  further  below,  TSAT  has  been 
shown  to  be  the  most  efficient  DTP  solver  previously  developed  [Armando,  Castellini  et  al. 
1999].)  The  goal  of  the  experiments  was  to  assess  the  effectiveness  of  various  combinations  of 
the  pruning  strategies  described  in  the  previous  sections.  As  is  customary  in  the  DTP  literature 
[Stergiou  and  Koubarakis  1998;  Armando,  Castellini  et  al.  1999;  Oddi  and  Cesta  2000;  Stergiou 
and  Koubarakis  2000],  experimental  sets  were  produced  using  the  random  DTP  generator  im¬ 
plemented  by  Stergiou,  in  which  DTPs  are  instantiated  according  to  the  parameters  <k,  m, 
E>,  where  k  is  the  number  of  disjuncts  per  constraint,  N  the  number  of  DTP  variables,  m  the 
number  of  DTP  constraints,  and  E  a  positive  integer  such  that  for  aU  the  disjuncts  x—j  —  h 
.  [0,  L]  with  uniform  probability.  In  the  random  DTP  problems  we  used,  we  used  the  typical 

settings  in  the  literature  where  /fe  =  2,  E  =  100,  and  JV .  {10,  15,  20,  25,  30,  35}.  Parameter  k  is 
chosen  to  be  2  because  this  is  the  case  for  constraints  that  typically  appear  in  many  planning 
and  scheduling  (e.g.  A<B  or  B<A).  We  also  employ  a  derived  parameter  R,  the  ratio  of  con¬ 
straints  over  variables,  mjN.  For  each  setting  of  JV,  we  varied  R  from  2  to  14,  and  we  gener¬ 
ated  50  random  problems  for  each  setting  of  N  and  R  (For  example,  we  generated  50  prob¬ 
lems  for  the  case  where  N  is  30  and  R  is  10;  those  problems  have  30  variables  and  300  con¬ 
straints).  The  total  number  of  experiment  problems  was  50  •  13  •  6  =  3900  (13  values  for  R,  6 
values  for  JV).  The  domains  of  the  variables  are  integers  instead  of  reals  so  that  semantic 
branching  can  easily  be  implemented:  the  negation  of  the  constraint  x  —j  =  b  is  j  —  x  —  -b  —1^^. 
This  is  again  standard  with  the  rest  of  the  literature. 

The  output  of  EpiHtis  provides  the  following  statistics  for  each  DTP  solved: 

The  Time  it  took  to  solve  the  problem. 

The  number  of  constraint  checks  CCs  (i.e.,  the  number  times  the  algorithm 
checked  the  FC-condition  or  the  Subsumption-Condition). 

The  number  of  search  Nodes  generated. 

The  number  of  constraint  propagations  CProps  (i.e.,  number  of  calls  to  maintain- 
consistency). 


13  There  are  specific  reasons  why  we  chose  to  implement  as  First,  if  the  variables  are  integer-valued  and  all 

the  bounds  are  integer  valued,  then  obviously  y-x<-b  is  equivalent  to  y-x<-b-l  which  is  stricter  than  y-x<-b-s.  In  addition, 
both  in  TSAT  and  in  the  Oddi  and  Cesta’s  work,  this  is  the  method  that  semantic  branching  has  been  implemented.  Thus,  it 
would  be  unfair  to  compare  Epilitis  with  TSAT  using  any  other  method.  If  the  assumption  of  integer  valued  variables  and 
bounds  does  not  hold,  semantic  branching  can  be  implemented  as  y-x<-b-s  or  even  y-x<-b  (which  is  not  as  strict  as  possi¬ 
ble,  but  sound). 
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The  number  of  no-goods  checks  ]VCs,(  i.e.  the  number  of  times  a  no-good  is 
checked  for  retrieval). 

The  number  of  no-goods  recorded  JVGs. 

In  the  graphs  and  tables  showing  the  results  below,  except  where  otherwise  noted  we  pre¬ 
sent  the  median  of  the  above  statistics  over  the  series  of  the  50  experiments  with  the  same  pa¬ 
rameters  N  and  R.  Again,  this  is  consistent  with  the  literature  on  DTP  solving. 

The  EpiHtis  algorithm  was  implemented  in  Allegro  Common  Lisp  5.0.  Both  EpiHtis  and  the 
ACG  solver  were  ran  on  the  same  Intel  Pentium  III  machine  running  Windows  2000,  having 
384MB  memory  and  a  clock  speed  of  IGHz.  There  is  no  time-out  for  the  experiments  run 
using  EpiHtis,  but  we  used  the  time-out  of  1000  seconds  provided  as  a  default  with  the  ACG 
solver.  This  time-out  Hmit  is  reasonable  because  it  is  an  order  of  magnitude  larger  than  the 
maximum  amount  of  time  taken  by  EpiHtis  to  solve  any  of  the  test  problems.  Note,  moreover, 
that  by  imposing  a  time-out  Hmit  on  ACG  but  not  on  EpiHtis,  we  are,  if  anything,  providing  an 
advantage  to  ACG  in  the  experimental  comparison. 

AH  of  our  experiments  confirm  the  existence  of  a  critical  region  for  values  R=5,  6^  7,  8, 
where  the  percentage  of  solvable  problems  is  less  than  10%  and  the  median  time  to  find  a  so¬ 
lution  or  prove  there  is  no  solution  to  a  DTP  problem  substantiaHy  exceeds  the  median  time 
taken  when  R<5  or  R>8. 

6.2  Pruning  Power  of  Techniques 

In  the  first  set  of  experiments  we  investigated  the  pruning  power  of  aU  the  pruning  meth¬ 
ods  and  combinations  thereof.  This  set  of  experiments  answers  the  foHowing  questions:  (i)  can 
aH  pruning  methods  be  integrated  efficiently?  (H)  how  do  the  pruning  methods  and  their  com¬ 
binations  compare  quantitatively? 

The  pruning  methods  we  tried  are: 

Removal  of  Subsumed  Variables  {RSVj 
Conflict-Directed  Backjumping  {CDB) 

Semantic  Branching  {SB) 

No-good  Recording  {NG) 

In  EpiHtis  aH  of  the  above  methods  can  be  individuaHy  turned  on  and  off,  providing  us  with 
the  opportunity  to  try  any  combination  we  desire.  The  only  Hmitation  is  that  whenever  NG  is 
on,  CDB  must  also  be  turned  on.  We  name  our  graphs  and  tables  using  the  foHowing  conven¬ 
tion:  we  Hst  the  options  that  were  turned  on  separated  by  spaces  or  dashes.  When  we  bound 
the  size  of  the  no-goods,  as  explained  in  Section  4.1,  we  foHow  the  name  with  the  numerical 
bound.  For  example,  CDB-RSV-SB-NG-10  is  EpiHtis  with  CDB,  RSV,  SB,  NG  on  and  a 
maximum  size  of  no-goods  set  to  10,  and  CDB-RSV-SB-NG  the  same  algorithm  with  no 
bound  on  the  size  of  no-goods.  We  use  the  term  ''Nothing”  to  identify  "bare”  EpiHtis,  with 
no  pruning  techniques  turned  on. 

For  this  set  of  experiments  we  used  the  following  variable  and  value  heuristics: 

Select  the  variable  according  to  the  MRV  (Minimum  Remaining  Values).  Break  the  ties 
by  selecting  the  variable  that  contains  the  value  that  maximi^^s  the  number  of  pair¬ 
wise  inconsistencies  with  the  values  in  the  domains  of  the  unassigned  variables. 
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Select  the  value  that  minimi^s  the  number  of  pairwise  inconsistencies  with  the  val¬ 
ues  in  the  domains  of  the  unassigned  variables. 

This  heuristic  is  typical  in  the  CSP  literature.  The  idea  is  that  by  choosing  the  variable  with 
the  value  that  maximizes  the  pairwise  inconsistencies,  the  branching  factor  of  the  search  is  re¬ 
duced,  since  this  is  the  variable  that  most  constrains  the  search.  On  the  other  hand,  when  we 
select  a  value  we  prefer  the  one  that  least  constrains  the  search  so  that  we  increase  the  prob¬ 
ability  of  finding  a  solution  that  contains  this  value. 


Ratio 

nothing 

rs 

cdb 

cdb 

rsv 

ng_10 

ng 

sb 

sb 

rsv 

cdb 

sb 

cdb 

sb 

rsv 

cdb 

sb 

rsv 

ng 

cdb 

rsv 

sb 

ng  10 

2 

0.02 

0.02 

0.02 

0.02 

0.03 

0.03 

0.02 

0.02 

0.02 

0.02 

0.03 

0.021 

3 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

0.05 

0.05 

0.05 

0.05 

0.06 

0.06 

4 

0.14 

0.13 

0.13 

0.11 

0.14 

0.15 

0.14 

0.12 

0.14 

0.11 

0.13 

0.13 

5 

1.91 

1.39 

1.05 

0.811 

0.49 

0.551 

0.531 

0.51 

0.501 

0.451 

0.421 

0.431 

6 

4.1 

3.33 

2.8 

2.39 

1.53 

1.46 

1.43 

1.34 

1.25 

1.24 

1.04 

0.941 

7 

1.93 

1.74 

1.87 

1.5 

1.07 

1.31 

1.05 

1.01 

0.981 

0.971 

1.02 

0.851 

8 

1.15 

1.11 

1.05 

0.892 

0.781 

0.982 

0.671 

0.651 

0.661 

0.621 

0.751 

0.701 

9 

0.711 

0.661 

0.671 

0.611 

0.621 

0.681 

0.551 

0.54 

0.521 

0.53 

0.62 

0.571 

10 

0.671 

0.611 

0.631 

0.571 

0.6 

0.66 

0.55 

0.551 

0.541 

0.511 

0.571 

0.531 

11 

0.56 

0.551 

0.521 

0.521 

0.541 

0.61 

0.461 

0.441 

0.451 

0.441 

0.531 

0.511 

12 

0.491 

0.501 

0.481 

0.461 

0.491 

0.551 

0.451 

0.431 

0.43 

0.431 

0.521 

0.48 

13 

0.461 

0.48 

0.461 

0.461 

0.461 

0.521 

0.45 

0.44 

0.42 

0.411 

0.51 

0.48 

14 

0.441 

0.42 

0.441 

0.43 

0.471 

0.541 

0.42 

0.41 

0.421 

0.411 

0.551 

0.491 

Table  2:  The  ordering  of  the  pruning  methods  according  to  Median  Time  when  R=6,  and 

N=20. 


Ratio 

rsv 

cdb 

cdb 

rsv 

ng_10 

sb 

ng 

cdb 

sb 

sb 

rsv 

cdb 

sb 

rsv 

cdb 

rsv 

sb 

ng_10 

cdb 

sb 

rsv 

ng 

2 

0.04 

0.04 

0.05 

0.04 

0.04 

0.05 

0.04 

0.04 

0.04 

0.05 

0.05 

3 

0.1 

0.11 

0.111 

0.12 

0.11 

0.12 

0.11 

0.1 

0.1 

0.11 

0.11 

4 

0.37 

0.291 

0.31 

0.32 

0.361 

0.311 

0.311 

0.32 

0.251 

0.29 

0.281 

5 

9.22 

4.3 

2.41 

1.06 

2.08 

0.892 

1.5 

1.69 

1.04 

0.811 

0.681 

6 

40.8 

27.5 

24.8 

10.1 

8.77 

8.72 

7.98 

7.7 

7.43 

7.05 

5.38 

7 

18.5 

16.3 

14.1 

8.56 

7.72 

7.66 

7.94 

7.47 

6.94 

6.78 

7.09 

8 

6.8 

5.81 

5.02 

4.1 

3.37 

3.83 

3.36 

3.2 

3.14 

2.74 

2.71 

9 

4.99 

4.46 

4.45 

3.51 

3.3 

3.56 

3.14 

3.1 

2.89 

2.83 

2.36 

10 

3.46 

2.69 

2.45 

2.08 

2.01 

2.36 

1.94 

1.97 

1.74 

1.92 

1.9 

11 

2.48 

2.21 

1.86 

1.9 

1.52 

2.17 

1.57 

1.42 

1.36 

1.61 

1.52 

12 

2.44 

2.3 

1.98 

1.71 

1.44 

1.86 

1.58 

1.36 

1.36 

1.53 

1.44 

13 

1.84 

1.63 

1.44 

1.31 

1.26 

1.48 

1.2 

1.13 

1.11 

1.18 

1.16 

14 

1.68 

1.3 

1.25 

1.38 

1.18 

1.51 

1.07 

1.12 

1 

1.3 

1.22 

Table  3:  The  ordering  of  the  pmning  methods  according  to  Median  Time  when  R=6,  and  N=25.  In 

Table  2,  Table  3,  and  Table  4  we  show  the  results  for  N=20,  N=25,  N=30  for  various  pruning 
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methods  and  their  combinations.  The  results  for  N<20,  not  reported  here,  are  similar.  The 
columns  are  listed  in  increased  order  of  efficiency  for  Ratio=6;  this  is  the  peak  of  the  critical 
region.  The  tables  are  not  complete,  i.e.,  some  pruning  method  combinations  are  missing,  be¬ 
cause  they  caused  the  algorithm  to  be  too  slow  for  the  experiments  to  complete  (e.g.  ''noth¬ 
ing”,  i.e.  the  no  pruning  methods  version  is  not  reported  in  Table  3).  All  times  are  reported  in 
seconds,  as  in  all  experiments  in  this  paper.  We  selected  size  10  for  bounding  the  no-good  size 
because  in  other  experiments  (described  subsequently),  size  10  was  determined  to  be  optimal 
size  for  the  EpiHtis  with  aU  pruning  methods  on.  Although  it  would  be  desirable  to  have  an 
analytic  technique  for  predicting  the  optimal  size  of  no-goods,  we  do  not  know  of  a  suitable 
such  account,  and  to  date,  aU  results  on  optimal  no-good  size  have  been  determined  experi- 
mentaly. 

As  expected  in  Tables  2-4,  "nothing”  performs  the  worst,  then  RSV,  then  CDB,  then  SB, 
NG,  and  NG-10  following  closely  together.  It  makes  sense  to  compare  the  time  of  each  algo¬ 
rithm,  since  their  underlying  implementation  is  the  same. 


Ratio 

sb 

cdb  sb 

rsv 

cdb  sb 

sb  rsv 

cdb  sb 

rsv 

ng_10 

2 

0.07 

0.07 

0.07 

0.07 

0.08 

3 

0.17 

0.18 

0.17 

0.17 

0.18 

4 

0.4 

0.39 

0.371 

0.36 

0.39 

5 

10.3 

7.42 

7.12 

8.25 

4 

6 

149 

142 

140 

138 

79.8 

7 

74.6 

70.5 

69.7 

78.2 

48.8 

8 

29.4 

26.6 

25.9 

31.5 

21.1 

9 

13.9 

12.6 

12.4 

14.1 

11.1 

10 

9.73 

9.15 

8.92 

10.3 

7.72 

11 

6.73 

6.28 

6.09 

6.69 

4.93 

12 

4.47 

4.64 

4.42 

4.61 

4.12 

13 

4.32 

4.31 

3.77 

4.38 

3.97 

14 

3.96 

4.02 

3.91 

4.17 

3.2 

Table  4:  The  ordering  of  the  pruning  methods  according  to  Median  Time  when  R=6,  and 

N=30 

Since  the  search  space  increases  exponentially  with  the  number  of  variables,  the  results  be¬ 
come  more  significant  as  N  grows.  Thus  the  differences  among  pruning  methods  is  most  ob¬ 
vious  in  Table  4,  which  shows  the  results  for  N=30.  The  worst  combination  is  the  CDB-RSV: 
we  were  not  even  able  to  complete  this  experiment  for  N=30.  The  best  performance  is  the 
CDB-SB-RSV-NG-10.  When  we  did  not  bound  the  size  of  the  no-goods,  the  performance  of 
the  algorithm  was  seriously  degraded  for  N=30  and  this  is  why  it  is  not  included  in  the  table. 

We  now  compare  the  different  pruning  methods  strictly  according  to  their  pruning  power, 
i.e.  not  including  the  computational  overhead  to  implement  them.  Table  5  shows  the  statistic 
Nodes J where  Nodes ^  is  the  median  number  of  search  nodes  explored  by  the  algo¬ 
rithm  in  each  column  c,  and  Nodes^^^^^-^^  the  search  nodes  explored  by  Epilitis  with  no  pruning 
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methods.  As  we  would  expect,  the  more  methods  we  add,  the  more  we  prune  the  search 
space.  In  this  case  NG  is  the  best  single^"^  method,  exploring  only  27.31%  of  the  whole  search 
space  (i.e.  when  no  pruning  method  is  on)  for  R=6.  NG  is  even  better  than  the  combination 
of  aU  three  other  methods  CDB-SB-RSV,  which  explores  32.52%  of  the  space.  This  result  en¬ 
courages  us  to  look  for  even  more  efficient  implementations  of  recording  and  retrieving  no¬ 
goods  to  reduce  the  overhead  of  the  technique.  Table  6  supports  the  same  argument,  showing 
the  effect  of  pruning  methods  on  the  search  space  explored  for  N=30,  where  the  search  space 
is  significantly  larger  than  for  N=20  (the  value  in  Table  5).  The  statistic  displayed  is 
N odes JN odes ,  where  Nodes ^  is  the  median  number  of  search  nodes  explored  by  the  algorithm 
in  each  column  and  Nodes^^  the  search  nodes  explored  by  Epiktis  with  SB  on.  For  example, 
in  the  last  column,  for  R=6,  we  see  that  the  ratio  is  38.99%  meaning  that  the  algorithm  CDB- 
RSV-SB-NG-10  explored  only  38.99%  of  the  space  the  algorithm  SB  explored  on  problems 
for  N=30  and  R=6.  The  results  show  in  an  impressive  way  the  pruning  power  of  no-goods: 
the  last  column,  corresponding  to  the  algorithm  with  the  no-goods  on,  display  a  significant 
reduction  of  the  space  searched. _ 


Ra¬ 

tio 

rsv 

cdb 

cdb  rsv 

sb 

sb  rsv 

cdb  sb 

cdb  sb 

rsv 

ng_10 

ng 

cdb  sb 

rsv 

ng_10 

cdb  sb 
rsv  ng 

2 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

3 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

100.00 

% 

4 

100.00 

% 

100.00 

% 

98.91% 

97.83% 

97.83% 

97.83% 

97.83% 

100.00 

% 

100.00 

% 

97.83% 

97.83% 

5 

76.00% 

42.00% 

42.00% 

30.40% 

30.40% 

27.73% 

27.73% 

20.33% 

17.53% 

15.67% 

15.13% 

6 

88.84% 

67.36% 

63.64% 

35.29% 

35.29% 

32.77% 

32.52% 

31.12% 

27.31% 

19.75% 

19.75% 

7 

89.71% 

79.22% 

73.69% 

50.49% 

50.49% 

45.24% 

44.66% 

39.13% 

39.13% 

31.17% 

32.82% 

8 

100.00 

% 

84.01% 

77.07% 

54.72% 

54.72% 

51 .25% 

50.67% 

52.99% 

53.95% 

44.51% 

44.12% 

9 

84.00% 

84.00% 

78.00% 

78.00% 

72.00% 

72.00% 

70.80% 

70.00% 

59.60% 

59.60% 

10 

95.24% 

81 .43% 

78.10% 

77.62% 

77.62% 

69.05% 

69.05% 

68.10% 

68.10% 

56.19% 

56.19% 

11 

98.57% 

81 .43% 

81 .43% 

77.14% 

77.14% 

67.86% 

67.86% 

71.43% 

71 .43% 

66.43% 

66.43% 

12 

100.00 

% 

96.12% 

96.12% 

95.15% 

95.15% 

86.41% 

86.41% 

84.47% 

84.47% 

78.64% 

78.64% 

13 

100.00 

% 

88.64% 

88.64% 

92.05% 

92.05% 

81 .82% 

81.82% 

84.09% 

84.09% 

79.55% 

79.55% 

14 

100.00 

% 

97.56% 

97.56% 

95.12% 

95.12% 

87.81% 

87.81% 

82.93% 

82.93% 

82.93% 

82.93% 

Table  5:  The  statistic  Median  Nodes  divided  by  Median  Nodes  of  “Nothing”  for  N=20.  The 
pruning  methods  are  sorted  according  to  this  statistic  for  R=6. 


Summarizing  the  results  of  this  section: 

A  rough  partial  ordering  of  the  pruning  methods  is  RSV  <  CDB  <  CDB-RSV  <  NG- 
10  <  SB  <  {CDB-SB,  SB-RSV,  CDB-SB-RSV}  <  CDB-SB-RSV-NG-10. 


14  Recall,  however,  that  when  NG  is  on,  CDB  is  also  on,  so  the  comparison  is  not  entirely  fair. 
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No-good  learning  needs  to  limit  the  size  of  the  no-goods  recorded  because  asymp¬ 
totically  the  overhead  of  recording  and  looking-up  all  the  possible  no-goods  greatly  out¬ 
weighs  the  benefits. 

SB  is  the  best  single  pruning  method  in  terms  of  performance,  i.e.  displaying  a  good 
trade-off  between  pruning  power  and  implementation  overhead. 

NG  is  the  best  single  pruning  method  in  terms  of  pruning,  even  better  than  all  the 
other  methods  combined  CDB-SB-RSV. 

The  Epilitis  with  options  CDB-SB-RSV-NG-10  considerably  improves  performance 
over  all  other  methods  combined  CDB-SB-RSV. 


Ratio 

sb  rsv 

cdb  sb 

cdb  sb 

rsv 

cdb  sb 
rsv  ng 

10 

2 

100.00% 

100.00% 

100.00% 

100.00% 

3 

100.00% 

100.00% 

100.00% 

100.00% 

4 

89.31% 

89.94% 

89.31% 

89.31% 

5 

92.96% 

71.37% 

69.25% 

32.16% 

6 

100.00% 

94.93% 

94.93% 

38.99% 

7 

100.00% 

97.13% 

93.03% 

47.13% 

8 

100.00% 

89.59% 

89.59% 

55.90% 

9 

100.00% 

86.26% 

86.26% 

66.79% 

10 

100.00% 

93.37% 

93.37% 

65.06% 

11 

100.00% 

85.98% 

85.98% 

60.31% 

12 

100.00% 

92.95% 

92.95% 

63.57% 

13 

100.00% 

95.23% 

95.23% 

72.57% 

14 

100.00% 

96.97% 

96.97% 

66.67% 

Table  6:  The  statistic  Median  Nodes  divided  by  Median  Nodes  of  SB  for  N=30.  The  pruning 
methods  are  sorted  according  to  this  statistic  for  R=6. 


6.3  The  Number  of  Forward-Checks  is  the  Wrong  Measure  of  Perform¬ 
ance 

Our  first  set  of  experiments  were  designed  to  compare  the  effectiveness  of  alternative 
combinations  of  pruning  strategies  and  it  was  straightforward  to  present  the  results  of  those 
experiments  since  what  we  are  concerned  with  is  precisely  the  number  of  search  nodes  in  the 
meta-CSP  that  are  pruned.  In  the  third  major  experiment,  which  we  present  below  in  Section 
6.5,  we  compare  EpiHtis  running  with  the  most  effective  combination  of  pruning  heuristics,  to 
TSAT,  the  previous  most  effective  DTP  solver.  It  is  less  obvious  what  metrics  to  use  in  this 
comparison.  It  is  customary  in  the  literature  to  compare  DTP  solvers  and  report  their  per¬ 
formance  using  the  number  of  forward-checks — more  precisely  the  total  number  of  values 
that  the  algorithm  forward-checked  during  search,  also  called  consistency  checks  CCs^ 


In  our  implementation  a  consistency-check  is  essentially  checking  the  FC-condition.  We  also  felt  that  we  should  count  as  a 
consistency-check  determining  whether  the  Subsumption-Condition  holds,  because  both  are  similar  operations  having  simi¬ 
lar  functions  and  take  the  same  time.  Not  counting  the  Subsumption-Condition  checks  in  CCs  would  favor  all  algorithms 
with  RSV  on  since  those  are  the  ones  that  perform  this  operation. 
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[Stergiou  and  Koubarakis  1998;  Armando,  CasteUini  et  al.  1999;  Oddi  and  Cesta  2000;  Stergiou 
and  Koubarakis  2000].  Such  comparisons  make  the  implicit  hypothesis  that  the  number  of 
consistency  checks  is  a  machine  and  implementation  independent  measure  of  performance.  In 
this  section  we  present  both  theoretical  arguments  and  empirical  evidence  that  this  hypothesis 
is  false  and  CCs  is  the  wrong  measure  of  performance. 

There  are  at  least  three  reasons  for  rejecting  CC  counts  as  a  performance  metric.  First,  the 
use  of  no-good  recording  in  EpiUtis  gives  an  unfair  advantage  for  a  comparison  based  on  CC 
counts.  This  is  because  no-good  recording  requires  significant  overhead  to  record  and  retrieve 
no-goods,  and  this  overhead  is  not  represented  in  the  number  of  forward-checks. 

Second,  as  discussed  in  Section  2.2,  the  time  required  for  each  consistency  check  depends 
on  the  method  used  for  maintaining  consistency.  For  example,  when  the  distance  array  of  each 
current  STP  is  available,  checking  the  FC-condition  takes  constant  time,  but  when  it  is  not, 
more  time  might  be  required. 

The  third  argument  against  the  use  of  CC-counts  as  a  metric  is  that  there  are  techniques 
that  have  been  employed  in  previous  DTP  solvers  that  result  in  fewer  values  being  forward 
checked  even  though  more  time  is  spent  exploring.  For  example,  a  technique  used  by  Stergiou 
and  Koubarakis  [Stergiou  and  Koubarakis  1998;  Stergiou  and  Koubarakis  2000]  and  ACG 
[Armando,  CasteUini  et  al.  1999]  is  what  we  wiU  caU  Forward-Checking  Switch-off  (FC-Of^.  In 
Appendix  C,  we  provide  an  example  that  iUustrates  that  that  FC-Off  can  reduce  the  number  of 
forward-checks  by  increasing  the  number  of  search  nodes  explored  and  thus  it  may  even  de¬ 
crease  the  performance  of  a  DTP  solving  algorithm. 

As  a  result  of  the  three  problems  with  using  CC  counts  as  a  measure  of  performance,  we 
report  actual  computation  times  used  in  our  comparison  of  TSAT  and  EpiUtis.  One  draw¬ 
back  of  such  a  comparison  is  that  we  cannot  as  easily  draw  conclusions  about  the  pruning  effi¬ 
ciency  of  EpiUtis’  additional  pruning  methods,  such  as  RSV,  CDB,  and  NG,  since  TSAT  uses  a 
very  inefficient  method  for  consistency  checks.  Thus,  the  better  performance  of  EpiUtis  might 
be  attributed  only  to  the  better  consistency  checking  techniques  it  employs.  We  cannot  totaUy 
dismiss  this  hypothesis  until  a  version  of  TSAT  is  re-implemented  using  better  forward-checks 
methods.  However,  our  first  set  of  experiments,  which  analyzed  the  pruning  methods  and 
showed  their  effectiveness,  make  it  unUkely  that  efficient  consistency  checking  is  solely  respon¬ 
sible  for  EpiUtis’  performance  advantage. 

6.4  Heuristics  and  Optimal  No-good  Size  Bound 

Before  presenting  the  actual  comparison  of  EpiUtis  and  TSAT,  we  need  to  pin  down  one 
more  detaU,  namely,  the  search  heuristic  used  for  selecting  which  variable/ value  combination 
to  select  during  each  stage  of  the  search.  Interestingly,  the  use  of  no-good  recording  increases 
the  range  of  search  heuristics  avaUable,  because  the  heuristic  itself  can  take  into  account  the 
no-good  information.  In  turn,  however,  this  means  that  the  performance  of  the  search  heuris¬ 
tic  is  intertwined  with  the  size  of  the  no-good  recorded.  Thus,  we  designed  a  factorial  experi¬ 
ment  aimed  at  discovering  the  best  combination  of  search  heuristic  (from  amongst  a  set  of 
plausible  heuristics)  and  bound  on  no-good  size. 

As  heuristic  information  we  considered  four  functions  that  estimate  how  much  a  value  x 
constrains  the  remaining  search  space.  These  are: 
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Figure  13:  Comparison  for  the  best  overall  algorithm  for  N=30. 

EO.  the  number  of  remaining  values  that  are  pairwise  inconsistent  with  x  (i.e.  calcu¬ 
lated  dynamically  during  search  using  the  current  domains). 

El :  the  number  of  values  that  are  pairwise  inconsistent  with  x  determined  staticallj 
before  search  begins. 

E2 :  the  number  of  the  remaining  values  that  are  pairwise  inconsistent  with  x  plus 
the  number  of  no-goods  x  appears  in. 

E3 :  the  number  of  the  remaining  values  that  are  pairwise  inconsistent  with  x.  Ties 
are  broken  by  the  number  of  no-goods  x  appears  in. 

As  estimators  of  how  much  a  variable  v  constrains  the  remaining  search  space  we  used  the 
maximum  of  the  value  estimator  used  over  all  remaining  values  in  ^’s  domain. 

In  each  of  our  heuristics  we  foUow  the  principle  of  selecting  the  variable  that  most  con¬ 
strains  the  remaining  search  space,  in  an  attempt  to  minimize  the  branching  factor.  Thus,  a 
variable  with  minimal  current  domain  is  chosen  first  (Minimum  Remaining  Values  heuristic)  by 
default.  However,  since  aU  domains  have  maximum  size  two,  it  is  often  the  case  that  there  are 
many  ties,  and  these  are  then  broken  by  using  one  of  the  above  estimators  E0-E3,  giving  rise 
to  the  four  heuristics  respectively. 

In  contrast,  as  value  selection  principle  we  select  the  value  that  least  constrains  the  search 
space,  in  an  effort  to  hit  a  solution  faster.  We  again  use  the  estimators  above.  For  example,  in 
H2  we  would  first  select  the  variable  v  with  the  value  that  maximizes  the  number  of  pairwise 
inconsistencies  and  participates  in  the  most  recorded  no-goods,  among  aU  the  variables  with 
least  domain  size.  Then,  we  would  select  the  value  in  ils  domain  that  minimizes  this  estimator 
{E2). 
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Figure  13  presents  the  results.  The  x-axis  shows  the  R,  ratio  of  constraints  to  variables;  this 
is  the  critical  parameter  for  the  DTP-solving  problems.  The  j-axis  shows  computation  time 
taken,  in  seconds;  note  that  the  scale  is  logarithmic.  We  name  the  curves  as  ''NG  xj”  to  de¬ 
note  EpiHtis  with  all  the  pruning  options  on^  where  x  is  the  Hmit  on  the  size  of  no-goods,  and  j  is 
the  search  heuristic  used.  We  show  only  the  graph  for  N=30  since  this  is  the  largest  size  we 
ran.  For  each  no-good  size  the  best  heuristic  is  selected  (e.g.  for  size  6  we  determined  the  best 
heuristic  to  be  H2).  Overall,  the  combination  of  ii^with  size  bound  of  10  works  best,  al¬ 
though  H3with  bound  of  14  come  very  close.  However,  we  suggest  using  ii^with  size  bound 
of  10  because  it  exhibits  a  better  average  case  behavior. 

To  summarize  the  results  of  this  section: 

The  recorded  no-goods  do  not  only  prune  the  search  space  but  can  also  be  effectively 
used  as  heuristic  information. 

EpiHtis  with  CDB,  SB,  RSV,  NG  on,  maximum  no-good  size  10,  and  heuristic  H2  h 
the  best  algorithm  in  the  set  of  experiments  we  ran. 

6.5  Comparing  EpiHtis  to  the  previous  state-of-the-art  DTP  solver 

In  Section  7  below  we  provide  details  of  EpiHtis’  three  predecessors:  one  developed  by 
Stergiou  and  Koubarakis  [Stergiou  and  Koubarakis  2000],  one  by  Oddi  and  Cesta  [Oddi  and 
Cesta  2000],  and  one  by  Armando,  CasteHini,  and  GiunchigHa  (TSAT)  [Armando,  CasteHini  et 
al.  1999].  Oddi  and  Cesta  present  an  experimental  comparison,  noting  that  whHe  their  algo¬ 
rithm  consistently  outperforms  that  of  Stergiou  and  Koubarakis,  it  is  at  best  competitive  with 
TSAT  algorithm.  Moreover,  they  show  that  TSAT  is  particularly  good  at  the  hardest  prob¬ 
lems,  i.e.,  those  in  the  critical  region.  As  Oddi  and  Cesta  note,  'Turther  work  [on  their  system] 
wiH  be  needed  to  clearly  outperform  TSAT.”  These  are  the  problems  that  are  most  significant, 
since  problems  outside  of  this  range  can  already  be  solved  very  quickly.  Given  these  results, 
we  view  TSAT  as  the  state-of-the-art  predecessor  to  EpiHtis,  and  conduct  head-to-head  ex¬ 
periments  with  it.  It  is  worth  noting,  however,  that  there  is  one  class  of  problems  for  which 
Oddi  and  Cesta’s  approach  outperforms  that  of  TSAT:  problems  with  smaH  R  values  (R  =  5). 
On  these  problems,  Oddi  and  Cesta’s  system  is  about  one  order  of  magnitude  faster  than 
TSAT.  Although  we  have  not  conducted  a  head-to-head  comparison  of  EpiHtis  with  Oddi 
and  Cesta’s  system  on  such  problems,  EpiHtis  is  also  about  an  order  of  magnitude  faster  than 
TSAT,  and  thus  it  is  reasonable  to  conclude  that  it  is  competitive  with  Oddi  and  Cesta’s  sys¬ 
tem  for  these  (relatively  easy)  problems. 

We  now  turn  to  the  detaHs  of  the  final  experiment,  in  which  we  compare  CDB-SB-RSV- 
NG-10-H2  with  TSAT.  Figure  14  shows  the  results  for  N=25  and  N=30.  As  explained  above, 
we  report  overaH  computation  time;  as  in  Figure  13,  the  x-axis  shows  R,  and  thej-axis,  which 
is  logarithmic,  shows  median  computation  time  in  seconds.  EpiHtis  is  faster  by  about  two  or¬ 
ders  of  magnitude  for  the  larger  (N=30)  problems.  Also  recaH  that  TSAT  has  a  time-out  of 
1000  seconds  imposed,  and  so  the  real  median  time  taken  by  their  program  might  be  signifi¬ 
cantly  more  than  is  indicated  here.  For  example,  for  N=30  and  R=6  the  median  time  is  exactly 
1000  seconds  implying  that  the  TSAT  solver  timed-out  on  more  than  half  the  problems. 
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We  also  ran  the  best  version  of  Epilitis  on  problems  of  up  to  size  N=35  (the  largest  size  of 
random  DTP  problems  reported  as  so  far)  and  we  observed  that  the  algorithm  scales  relatively 
well.  Figure  15  shows  the  performance  of  EpiHtis  on  problems  of  different  sizes.  For  the  larger 
problems  where  N=35  Epilitis  has  a  median  time  performance  of  about  100  seconds,  while 
the  corresponding  TSAT  performance  is  more  than  1000  seconds  for  problems  of  size  N=30. 
The  overall  performance  of  TSAT  is  also  shown  in  the  same  figure.  As  we  can  see  TSAT  re¬ 
quires  about  one  order  of  magnitude  more  time  every  time  N  is  increased  by  5.  In  contrast 
EpiHtis’  performance  does  not  degrade  as  fast. 

Summarizing  the  results  of  this  section: 

EpiHtis  is  almost  two  orders  of  magnitude  faster  than  the  previous  state-of-the-art 

DTP  solver  TSAT  on  standard  benchmark  problems. 

EpiHtis’  performance  scales  comparatively  weH  as  the  size  of  the  problems  increase. 
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6.6.  Application  Experience 


The  previous  sections  have  described  controlled  experiments  we  performed  using  synthetic, 
abstract  test  problems,  to  analyze  the  performance  of  the  various  pruning  strategies  developed 
for  DTP  solving  and  to  compare  Epilitis  with  the  previous  state-of-the-art  system.  We  also 
have  some  experience  using  Epilitis  in  a  large  application,  which  we  briefly  describe  here. 

Autominder  [Pollack  2002]  is  an  intelligent  cognitive  orthotic  system:  a  system  designed  to 
help  older  adults  with  memory  decline  by  providing  them  with  adaptive,  personalized  remind¬ 
ers  about  their  daily  activities.  Autominder  has  three  main  components:  a  Plan  Manager, 
which  stores  a  plan  of  it’s  clients  daily  activities,  and  is  responsible  for  updating  it  and  identify¬ 
ing  potential  conflicts  in  it;  a  Client  Modeler,  which  uses  information  about  the  client’s  observ¬ 
able  activities'*^  to  track  the  execution  of  the  plan,  inferring  what  activities  the  client  has  already 
performed  and  what  activities  are  pending;  and  a  Personal  Cognitive  Orthotic,  which  reasons 
about  any  disparities  between  what  the  client  is  supposed  to  do  and  what  she  is  doing,  and 
makes  decisions  about  when  to  issue  reminders. 

The  relevant  component  for  the  current  paper  is  the  Plan  Manager.  It  maintains  a  model  of 
the  cHent’s  daily  plan  encoded  as  a  DTP.  It  then  invokes  EpiHtis  to  update  the  plan  in  re¬ 
sponse  to  four  types  of  events: 

(1)  The  addition  of  a  new  activity  to  the  plan. 

(2)  The  modification  or  deletion  of  an  activity  in  the  plan. 

(3)  The  execution  of  an  activity  in  the  plan,  as  reported  by  the  CHent  Modeler. 

(4)  The  passage  of  a  time  boundary  in  the  plan. 

In  each  of  these  cases,  the  Plan  Manager  formulates  a  DTP:  for  instance,  when  a  new  ac¬ 
tivity  is  added,  the  DTP  consists  of  the  constraints  in  the  original  cHent  plan,  the  constraints  in 
the  new  activity,  and  a  set  of  constraints  generated  to  represent  the  resolution  of  any  conflicts 
between  the  two.  EpiHtis  then  attempts  to  solve  the  DTP,  indicating  whether  or  not  it  suc¬ 
ceeded,  and  returning  any  newly  required  constraints.  For  instance,  the  addition  of  an  activity 
may  result  in  added  constraints  on  the  time  of  performance  of  a  previously  added  activity. 

In  general,  the  size  of  the  plans  managed  by  Autominder  are  smaU  by  the  standards  of  the 
planning  community.  (On  the  other  hand,  our  focus  is  not  on  plan  generation,  but  on  a  range 
of  other  tasks,  such  as  plan  monitoring,  update,  and  dispatch.)  Generally  the  cHent  plan  has  on 
the  order  of  30  actions,  meaning  that  there  are  60  events  (start  and  end  points),  and,  with  typi- 
caUy  temporal  constraints,  the  representation  requires  about  4  or  5  constraints  per  action,  and 
about  2  or  3  disjuncts  per  constraint.  For  problems  of  this  scale,  EpiHtis  nearly  always  pro¬ 
duces  solutions  (or  determines  inconsistency)  in  less  than  a  second,  an  amount  of  time  that  is 
weH  within  the  bounds  we  require. 


The  current  version  of  Autominder  is  deployed  on  a  mobile  robot,  and  uses  on-board  sensors  to  observe  a  client’s  move¬ 
ment  about  her  home. 
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Technique 

SK 

ACG 

OC 

EpiHtis 

Temporal  Reasoning 

Constraint  Propa¬ 
gation 

Maintain  IDPC, 

o(|f|2) 

N/A 

Maintain  IFPC, 
0(|F|2) 

Maintain  IFPC, 
0(|F|2) 

Forward- 
Checking  (time 
complexity  is  for 
each  value) 

Find  distance,  check 
FC-Condition.  Ac¬ 
tual  implementation 

0(|K|  2);  could  be 
done  in  0(|K|  ). 

Run  an  STP  consis¬ 
tency  checking 
algorithm.  Actual 
implementation: 

0(2  ).  Could  be 

done  in  0(|j^| 

Lookup  distance, 
check  FC- 
Condition,  0(1) 

Lookup  distance, 
check  FC- 
Condition,  0(1) 

Value  Sub¬ 
sumption  (time 
complexity  is  for 
each  value) 

Not  in  the  original 
implementation. 
Could  be  done  by 
finding  distance, 
then  checking  Sub¬ 
sumption-Con¬ 
dition,  o(|f|  ) 

Not  in  the  original 
implementation. 
Could  be  done  by 
running  an  STP 
consistency  algo¬ 
rithm  0(F  3). 

Lookup  distance, 
check  Subsump¬ 
tion-Condition, 

0(1) 

Lookup  distance, 
check  Subsump¬ 
tion-Condition, 

0(1) 

Searching  Methods 

CDB 

Yes 

No 

No 

Yes 

RSV 

No 

No 

Yes 

Yes 

SB 

No 

Yes 

Yes 

Yes 

FC-off 

Yes 

Yes 

No 

Yes 

NG 

No 

No 

No 

Yes 

IFC19 

No 

No 

Yes 

No 

Heuristics 

Variable 

MRV 

Max-Inc 

MRV 

See  Section  6.3 

Value 

The  value  with  the 
time-points  with  the 
most  appearances  in 
other  values 

The  value  with  the 
most  pairwise  in¬ 
consistencies  (but  it 
might  be  negated) 

No  specific  heu¬ 
ristic 

See  Section  6.3 

Table  7:  Comparison  of  all  DTP  solvers 


The  time  complexity  shown  is  for  implementing  forward  checking  with  the  best  known  algorithm:  Given  that  in  the  SK 
approach  STP  ,  the  current  STP  is  in  directional  path  consistency,  we  can  find  the  distance  between  a  pair  of  nodes  in  time 

0(|K|  )  and  check  the  FC-Condition.  As  already  mentioned,  in  the  actual  implementation,  SK  used  a  less  efficient  scheme 

that  takes  quadratic  time.  In  the  ACG  approach,  we  have  to  run  an  STP  consistency  checking  algorithm,  while  in  the  actual 
implementation  ACG  use  Simplex. 

Neither  SK  nor  ACG  use  RSV,  so  these  two  cells  do  not  refer  to  their  actual  implementation.  Instead,  this  is  the  most  effi¬ 
cient  way  they  could  have  implemented  RSV  had  they  desired  to,  given  the  way  they  perform  temporal  propagation. 

19  IFC  refers  the  to  Incremental  Forward  Checking  technique  of  [Oddi  and  Cesta  2000]  in  which  a  value  v  \  x-j  =  is  for¬ 
ward-checked  only  if  the  distance  d^y  has  changed  since  last  forward  checking  took  place.  If  it  has  not,  then  v  satisfies  the  FC- 
Condition  for  sure.  We  mention  this  technique  for  completeness.  IFC  does  not  reduce  the  search  space  and  it  can  be  used  in 
conjunction  with  any  pmning  technique. 
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7.  Related  Work 


7.1  Previous  DTP  Solvers 

There  are  two  previous  DTP-solvers  that  treat  component-STP  selection  as  CSP  problems 
and  perform  a  search  in  the  same  meta-CSP  as  Epilitis:  that  of  Stergiou  and  Koubarakis 
[Stergiou  and  Koubarakis  1998;  Stergiou  and  Koubarakis  2000]  (hereafter  SK)  and  of  Oddi 
and  Cesta  [Oddi  and  Cesta  2000]  (hereafter  OC).  We  can  compare  these  approaches  in  terms 
of  the  implementation  of  (1)  maintain-consistency,  (2)  forward-check,  (3)  the  variable  or¬ 
dering  heuristic,  (4)  the  value  ordering  heuristic,  (5)  and  the  techniques  employed  for  the 
search  and  for  pruning  the  search  space.  An  additional  approach,  which  casts  DTP-solving  in 
terms  of  SAT,  is  discussed  in  Section  7.1.2. 

7.1.1  The  CSP  Approaches 

In  the  SK  approach,  function  forward-check  is  implemented  by  adding  a  value  to  the  cur¬ 
rent  STP  S  and  propagating  using  again  the  IDPC  algorithm,  identifying  the  inconsistency  if 
there  is  one,  and  then  retracting  the  constraint  using  again  IDPC  so  as  to  be  ready  to  forward 
check  the  next  value.  This  requires  two  calls  to  IDPC  with  0(|E|^)  in  the  worst-case  for  each 
value  to  be  forward  checked.  There  is  of  course  a  much  faster  algorithm:  checking  if  the  FC- 
Condition  holds.  The  FC-condition  requires  the  distance  between  two  time  points,  which 
given  that  the  SK  approach  maintains  the  current  STP  in  directed  path  consistency  form,  can 
be  found  in  0(|F|)  time  in  the  worst  case  with  the  algorithm  described  at  [Chleq  1995]. 

The  variable  ordering  heuristic  in  SK  is  the  Minimum  Remaining  Values  {MRV)  in  which  the 
variable  with  the  fewest  remaining  values  in  its  domain  is  selected  first.  Ties  among  variables 
are  broken  by  choosing  the  variable  that  contains  the  time-points  that  appear  the  most  in  the 
rest  of  the  variables.  Each  variable  may  contain  many  disjuncts  and  each  disjunct  contains  two 
time-points,  so  we  can  choose  the  variable  that  contains  the  time-point  with  the  maximum  ap¬ 
pearance  in  other  variables/disjunctions  or  the  variable  with  the  largest  sum  of  appearances  of 
its  time-points  in  other  variables /disjunctions.  The  SK  paper  does  not  discuss  exactly  how  the 
selection  is  performed.  The  value  ordering  heuristic  is  the  same  as  the  tiebreaker  heuristic 
above.  The  disjunct  whose  time-points  appear  in  the  most  in  other  variables  is  selected  first. 
SK  experimented  with  the  anti-heuristic  but  with  disappointing  results. 

For  the  approach  of  OC,  the  table  summarizes  well  the  design-choices  made.  The  OC  vari¬ 
able  ordering  heuristic  is  the  MRV  with  no  other  tie-breaking  heuristics.  There  is  no  particular 
value  ordering  heuristic. 

7.1.2  The  SAT  Approach 

The  Armando,  CasteUini,  and  GiunchigHa  (ACG)  approach  differs  from  the  previous  ones 
in  that  it  treats  the  component-STP  selection  not  as  a  meta-CSP  problem,  but  as  a  SAT  prob¬ 
lem  instead.  However,  we  can  stiU  use  the  above  classification  scheme  to  compare  the  ap¬ 
proach:  The  ACG  algorithm  does  not  use  maintain-consistency.  Every  time  the  algorithm 
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requires  checking  the  consistency  of  a  set  of  STP-like  constraints  they  use  a  version  of  the 
Simplex  algorithm  for  linear  programming.  Simplex  has  exponential  worst-case  performance. 

During  a  preprocessing  step,  ACG  enhances  the  SAT  formula  with  clauses  (equivalently 
variables  in  a  CSP-based  approach)  that  correspond  to  inconsistent  pairs  of  literals  (equiva¬ 
lently  values  in  a  CSP-based  approach).  This  provides  additional  guidance  to  an  MRV-like  cri¬ 
terion  for  variable  selection:  they  choose  the  clause  that  contains  the  literal  with  the  greatest 
number  of  occurrences  in  the  clauses  of  minimal-length  (which  implies  they  will  choose  the 
clause  with  the  literal  that  participates  in  the  most  pairwise  inconsistencies  with  other  literals). 
We  will  call  this  heuristic  Max-Inc  because  essentially  it  picks  the  clause  with  the  literal  that 
participates  in  most  pairwise  inconsistencies. 

For  variable  ordering,  they  choose  the  literal  that  maximizes  the  number  of  pairwise  incon¬ 
sistencies  in  the  previous  step.  Notice  here  however,  that  a  SAT-based  procedure  can  either 
choose  to  branch  on  the  literal  c-j  or  the  literal  .  Instead,  a  CSP-based  approach  can  only 
branch  on  c-j  ,  i.e.  assign  a  value  to  a  variable,  and  never  the  negation  of  the  value  to  a  variable. 
From  the  ACG  paper  it  is  unclear  which  branch  (i.e.  the  positive  or  the  negative)  is  taken  first 
and  how  this  choice  is  made. 

Table  7  summarizes  the  above  discussion  and  comparison  between  the  different  ap¬ 
proaches.  The  DPT  solving  approaches  are  ordered  chronologically  according  to  the  date  of 
appearance  in  the  literature. 

7.2  Other  Temporal  Reasoning  formalisms 

The  focus  of  this  paper  has  been  on  developing  more  efficient  techniques  for  solving 
DTPs.  One  question  we  must  address  is  why  we  want  to  use  DTPs,  when  there  are  other  for¬ 
malisms  for  temporal  reasoning  that  are  computationally  more  tractable.  For  example,  as  we 
noted  at  the  beginning  of  the  paper,  DTPs  subsume  both  Simple  Temporal  Problems  (STPs) 
and  Temporal  Constraint  Satisfaction  Problems  (TCSPs).  STPs  allow  only  non-disjunctive 

constraints,  while  TCSPs  allow  constraints  of  the  form  c.^ .  c.^  where  each  c-j  is  of  the 

form  X  —j  —  b  with  the  restriction  that  x  and  j  are  the  same  in  aU  c-j  .  STPs  can  be  solved  in 
polynomial  time.  Although  the  same  is  not  true  for  TCSPs,  they  are  stiU  computationally  more 
tractable  than  DTPs,  because  all  their  constraints  are  binary,  involving  only  two  variables, 
while  the  DTP  constraints  may  be  non-binary,  and  it  is  significantly  easier  to  calculate  path- 
consistency  in  networks  of  binary  constraints  than  in  networks  where  the  constraints  are  non¬ 
binary  [Bessiere  and  Regin  1997;  Bessiere  1999;  Bessiere,  Mesequer  et  al.  1999]. 

It  turns  out,  however,  that  the  limitations  of  TCSPs  do  result  in  weak  expressive  power:  in 
particular,  we  consider  the  most  serious  disadvantage  of  TCSPs  to  be  their  inability  to  express 
the  fact  that  two  intervals  should  not  overlap.  This  kind  of  constraint  is  essential  in  scheduling 
and  planning  applications  where  it  is  often  the  case  that  some  actions  should  not  overlap,  for 
example  if  they  utilize  the  same  unary  resource.  As  was  illustrated  in  the  example  in  the  Intro¬ 
duction,  it  is  straightforward  to  encode  such  a  restriction  with  a  DTP  constraint:  if  denote  with 
(A^  and  (B^  the  start  (end)  times  of  actions  xl  and  B  ,  then  the  fact  that  they  cannot 
overlap  can  be  written  as  the  DTP  constraint: 

Aj,-B^  =  0.  Bj,-A^  =  0. 
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This  constraint  involves  four  time-points  ,  A^ ,  and  and  so  it  cannot  be  represented 

by  a  TCSP,  but  it  is  perfectly  acceptable  in  a  DTP.  There  are  other,  binaijj,  representations 
however,  that  allow  such  constraints  to  be  represented.  These  include  the  Point-Interval  Algebra 
(PIA)  described  in  [Meiri  1991].  In  PIA  the  variables  can  be  either  time-points  or  intervals;  and 
aU  relations  are  binary:  interval-interval,  point-point,  interval-point.  The  constraints  between 
time-points  can  be  metric  TCSP  constraints,  while  the  rest  of  constraints  are  qualitative.  Hav¬ 
ing  two  more  interval  variables  Aj  and  Bj  representing  the  intervals  associated  with  the  actions 
can  then  encode  the  above  situation  by  imposing  the  disjunctive  constraint: 

Aj  {before,  after]  Bj 

Although  PIA  can  thus  model  prohibited  overlaps,  it  cannot  readily  handle  requirements  of 
temporal  separation  between  actions.  For  example,  suppose  that  H  and  B  are  two  medical 
treatment  procedures  appHed  to  the  same  patient  with  the  constraint  that  if  H  is  appHed  first,  B 
can  only  be  appHed  3  days  later,  while  if  B  is  performed  first  then  H  can  be  performed  2  days 
later.  The  constraint  cannot  be  represented  in  PIA  but  written  as  the  DTP  constraint 

Nevertheless,  extensions  of  the  PIA  have  appeared  that  aUow  constraints  of  this  sort  to  be 
represented  while  remaining  within  the  realm  of  binary  constraints  [Cheng  and  Smith  1995; 
Cheng  and  Smith  1995]. 

The  above  argument  may  suggest  that  we  can  avoid  non-binary  constraints  if  we  employ 
both  intervals  and  time-points  as  our  representational  elements.  However,  there  are  other 
types  of  constraints  that  are  inherently  non-binary  such  as  conditional  constraints  of  the  form 
'Hf  constrainti  then  constraint2^\  e.g.  'Hf  treatment  H  does  not  last  enough,  then  perform  treat¬ 
ment  B  for  at  least  e  days.”  The  constraint  can  be  written  as: 

<— (  d  =  A^  —  A^ )  ^  (e  =  B^  —  B^),  or  equivalently 
(d  >  A^  —  A^ )  .  (e  =  B^  —  B^ 
and  these  are  only  expressible  with  DTPs. 

There  are  two  other  formaHsms  that  are  as  expressive  as  DTPs,  namely  the  GeneraHzed 
Temporal  Network  (GNC)  described  in  [Staab  1998]  and  the  Temporal  Constraint  Networks 
(TCN)  of  Barber  [Barber  2000].  The  former  is  essentiaUy  a  DTP-Hke  formaHsm  that  aUows 
conjunctions  of  STP-Hke  constraints  in  each  disjunct.  Because  it  is  straightforward  to  convert  a 
constraint  of  this  form  into  a  standard  DTP-constraint,  the  advantages  of  GNCs  is  not  obvi¬ 
ous.  Barber’s  TNCs  are  also  as  expressive  as  DTP.  A  TNC  is  a  TCSP  with  the  addition  of 
what  Barber  caUs  I-L-Sets.  I-L-Sets  (Inconsistent-Label-Sets)  are  essentiaUy  no-goods:  an  I-L- 

Set  looks  has  the  form  .  denoting  that  the  conjunction  does  not  hold  in  the 

TCSP.  A  constraint  a  .  b  ,  where  a  and  b  are  STP-Hke  constraints  of  the  form  x  —j  =  b-j  and  w 
~  Z  ~  ^2  (involving  two  pairs  of  different  variables)  cannot  be  represented  as  a  TCSP  con¬ 
straint,  but  it  can  be  encoded  as  the  I-L-Set  <{<  a  .  <r-d)  (using  De’Morgan’s  rule  for  Boolean 
Algebra).  However,  TCNs  require  that  the  disjuncts  in  an  I-L-Set  participate  in  some  other 
TCSP  constraint.  Thus,  it  is  not  enough  to  just  add  the  above  I-L-Set;  we  also  have  to  add 
TCSP  constraints  so  that  a  and  b  appear  in  the  underlying  TCSP.  For  example,  we  can  add  the 
TCSP  constraints  {a  .  <r-d)  and  {b  .  Ad).  By  using  this  scheme,  TNCs  reach  the  expressiveness 
of  the  DTP,  albeit  in  a  pecuHar  way.  To  solve  TCNs,  Barber  in  [Barber  2000]  provides  a  path-1 
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consistency  algorithm  that  in  essence  calculates  the  full  set  of  all  no-goods  (whose  number  is 
exponential  to  the  number  of  disjuncts),  but  no  experimental  results  are  provided. 

7.2  Scheduling  algorithms 

Another  related  area  of  work  is  that  of  Automated  Scheduling.  In  particular,  the  Precedence 
Constraint  Posting  (PCP)  technique  of  Cheng  and  Smith  [Cheng  and  Smith  1995;  Cheng  and 
Smith  1995]  bears  certain  similarities  to  DTP  solving.  Cheng  and  Smith  apply  PCP  to  typical 
scheduling  problems  such  as  the  Job-Shop  Scheduling  QSSP)  and  the  Hoist  Scheduling  Prob¬ 
lem  with  very  encouraging  results.  Cheng  and  Smith  used  a  formalism  based  on  the  PIA  in 
[Meiri  1991]  and  employed  domain-specific  heuristics.  What  makes  DTP  and  PCP  solving 
similar,  and  distinguishes  them  from  most  other  automated  scheduling  techniques,  is  their  use 
of  the  meta-CSP  approach.  This  contrasts  with  scheduling  algorithms  that  formulate  the  prob¬ 
lem  as  a  CSP  with  variables  that  are  the  start  times  of  the  events,  and  directly  solve  that  CSP. 

Stergiou  and  Koubarakis  [Stergiou  and  Koubarakis  2000]  applied  their  DTP  solver  on  JSSP 
with  somewhat  disappointing  results.  However,  it  bears  remembering  that  they  were  compar¬ 
ing  a  fairly  general-purpose  temporal  reasoning  module  (their  DTP  solver)  against  many  highly 
optimized  algorithms  that  had  been  tuned  specifically  for  job-shop  scheduling  problems.  It  is 
also  worth  noting  that,  JSSP  problems  are  typically  optimization  problems:  it  is  often  relatively 
easy  to  find  a  solution,  but  very  hard  to  find  an  optimal  solution.  In  contrast,  at  least  on  the 
random  DTP  problems  we  have  tested,  just  finding  one  solution  is  inherently  hard. 

8.  Discussion,  Contributions,  and  Future  Work 

In  this  paper,  we  have  focused  on  the  development  of  efficient  techniques  for  solving  Dis¬ 
junctive  Temporal  Problems  (DTPs).  DTPs  are  a  class  of  constraint-based  temporal  reasoning 
problems  that  appeared  in  the  literature  for  the  first  time  only  in  1998  [Stergiou  and  Kouba¬ 
rakis  1998].  Although  DTPs  are  potentially  very  useful  for  a  range  of  planning  and  scheduling 
problems,  solving  them  can  be  computationally  quite  costly.  We  therefore  examined  the  strate¬ 
gies  that  had  been  proposed  in  the  previous  literature  for  improving  the  efficiency  of  DTP- 
solving,  considered  how  to  integrate  these  strategies  with  one  another  and  with  no-good  learn¬ 
ing,  and  conducted  systematic  experiments  to  determine  what  combination  of  strategies  is 
most  effective. 

Our  experiments  were  conducted  using  a  DTP-solving  system  that  we  implemented, 
EpiHtis.  EpiHtis  is  instrumented  so  that  the  user  can  ''turn  on”  various  pruning  strategies.  It  is 
pubHcly  available^^,  and  may  be  used  as  a  testbed  for  further  exploration  of  DTP  solving.  In 
our  own  experiments,  we  were  able  to  achieve  a  speed-up  of  almost  two  orders-of-magnitude 
over  the  previous  fastest  algorithm,  by  combining  a  set  of  pruning  strategies,  adding  no-good 
learning  with  an  experimentally  determined  size  bound,  and  using  a  carefully  analyzed  search 
heuristic. 

One  important  result  of  our  experiments  was  the  discovery  that  no-good  learning  is  par¬ 
ticularly  powerful  in  improving  the  efficiency  of  DTP-solving.  We  can  explain  this  by  noting 

20  Contact  the  first  author  at  ioannis.tsamardinos@vanderbilt.edu.  Epilitis  will  also  be  available  on  the  first  author’s  web  site  in 
the  future. 
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that,  in  a  DTP  solver,  forward-checking  a  value  requires  the  propagation  of  the  corresponding 
STP-constraint  in  the  current  STP,  which  is  a  relatively  costly  (albeit  polynomial)  operation. 
Thus,  even  though  there  is  computational  overhead  associated  with  retrieving  no-goods,  this 
overhead  may  be  outweighed  by  the  savings  in  forward-checking.  In  an  ordinary  (non¬ 
temporal)  CSP,  forward-checking  may  be  less  expensive,  and  the  benefits  of  no-good  re¬ 
cording  might  not  be  as  great. 

We  have  already  demonstrated  the  practical  usefulness  of  DTP-solving  in  general,  and 
EpiHtis  in  particular,  in  two  of  our  other  research  projects.  In  one  of  these,  the  Plan  Manage¬ 
ment  Agent  (PMA)  [PoUack  and  Horty  1999],  we  are  designing  an  intelligent  calendar  that 
manages  a  user’s  plan.  In  the  other,  the  Autominder  [Tsamardinos,  Vidal  et  al.  2002],  we  are 
designing  a  cognitive  orthotic  system  intended  to  manage  and  monitor  the  daily  activities  of  an 
elderly  user,  providing  him  or  her  with  appropriate,  timely  reminders.  EpiHtis  plays  a  central 
role  in  both  systems,  serving  as  the  main  engine  for  updating  and  modifying  the  modeled 
plans. 

There  are  many  avenues  for  future  work  on  this  topic,  and  here  we  mention  just  a  few. 

Explore  dynamic  DTPs.  Dynamic  DTPs  are  sequences  of  DTPs  that  differ  from  suc¬ 
cessive  elements  by  a  few  constraints.  Such  sequences  arise  naturaUy  in  certain  planning 
problems  in  which  new  goals  are  periodicaUy  added  to  an  existing  set  of  goals.  No-good 
recording  may  be  especiaUy  useful  for  dynamic  DTPs,  as  it  is  possible  that  some  of  the 
no-goods  identified  and  recorded  for  the  previous  DTP  wiU  stiU  hold  for  the  next  DTP 
in  the  sequence,  and  thus  can  prune  the  search  space.  In  one  extreme  case,  if  the  no¬ 
good  <— I ,  J>  StiU  holds  in  the  next  DTP,  then  the  DTP  is  inconsistent  and  this  is 
proven  without  any  search  performed!  In  [Schiex  and  VerfaiUie  1994]  it  is  shown  that 
the  method  greatly  improves  the  performance  of  CSP  solvers  on  dynamic  (non¬ 
temporal)  CSPs. 

Consider  replacing  forward-checking  as  the  underlying  algorithm  in  EpiHtis  with  a  fuU 
looking  ahead  approach,  investigating  the  interactions  between  such  an  approach  and 
the  various  speed-up  techniques  discussed  in  this  paper. 

Investigate  the  use  of  random  restart  techniques  [Gomez,  Selman  et  al.  1998]  to  sup¬ 
plement  the  efficiency-increasing  strategies  already  described  in  this  paper.  It  seems 
plausible  that  there  wiU  be  a  synergistic  influence  between  no-good  recording  and  ran¬ 
dom  restarts.  We  also  expect  that  certain  scheduling  techniques,  such  as  profiling,  could 
be  appUed  to  DTP  solving. 

Use  DTPs  to  model  conditional  plans.  As  mentioned  in  Section  7.1,  we  noted  that  the 
n-ary  constraints  of  DTPs  can  be  used  to  model  plans  with  conditional  (if-then) 
branches.  In  work  already  underway,  we  have  analyzed  the  notion  of  consistency  in 
conditional  temporal  networks,  and  have  shown  that  consistency  checking  in  these 
networks  can  be  reduced  to  DTP-solving  [Tsamardinos,  Vidal  et  al.  2002]. 

Introduce  temporal  uncertainty.  Even  though  DTPs  are  very  expressive,  subsuming  a 
large  number  of  other  temporal  and  scheduling  problems,  they  have  one  key  Hmitation: 
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Figure  16:  Proof  of  Theorem  1 

they  do  not  readily  permit  one  to  model  events  whose  time  of  occurrence  is  not  known 
and  is  not  under  the  control  of  a  planning  agent.  The  recently  developed  STPU  formal¬ 
ism  does  support  modeling  of  such  events,  but  only  as  extensions  to  STPs.  It  would  be 
very  useful  to  develop  techniques  for  combining  STPU  and  DTPs. 


Appendix  A:  Proofs  of  Theorems 

Let  us  denote  by  d^j(S)  the  distance  between  time-points  x  andj/  in  STP  S  and  by  d^j(A) 
the  distance  between  time-points  x  andj  in  the  STP  induced  by  assignment  xl. 

Theorem  1:  A  value  cij  \j  — x  =  bxj  is  inconsistent  with  a  consistent  STP  S  (that  is,  S 
.  cij  is  inconsistent)  if  and  only  if  the  following  condition  holds: 

+  dy^{S)  <  0  (FC-condition) 

Proof:  Let  be  the  shortest-path  from  j  to  x  in  L  and  thus  the  length  of  is  d^^.  If 
the  FC-condition  holds,  then  the  path  p^^  .  (xj)  has  length  +  d^JS)  and  so  it  forms 
a  negative  cycle  making  S  inconsistent.^^ 

Conversely,  let  us  suppose  that  the  FC-condition  is  false  and  prove  that  —  S  .  c-j 
will  be  consistent.  We  will  prove  this  claim  by  contradiction,  i.e.  we  will  assume  FC 
condition  is  false  and  LUs  inconsistent,  and  will  derive  a  contradiction.  If  LUs  incon¬ 
sistent  then  there  is  a  negative  cycle,  and  because  S  was  consistent  before  we  added  c-j  , 
that  means  that  the  negative  cycle  involves  the  new  constraint  c-j  .  Let  us  assume  the 
negative  cycle  is  c-j  .  for  some  path  .  So  +  d”y^  =  +  iength(p’^J  <  0  (1), 

where  d’^^^  is  the  distance  between j  and  x  in  for  some  path p’^ ,  which  is  the  shortest 
path  from  J  to  x  (Figure  16).  Since  the  negative  cycle  is  a  simple  cycle  (no  loops  al¬ 
lowed),  then  edge  (x,  j)  is  not  a  member  oip’\  Therefore,  the  shortest  path p’’  fromj 
to  X  in  LAs  does  not  contain  the  new  constraint  c-j  and  thus  distance  fromj  to  x  in  L' 
and  S  is  the  same:  d^^  —  d’’^^  .  By  (1)  above  we  get  that  b^^  +  d’’^^  —  b^y  +  d^^  <0,  i.e.  FC- 
condition  holds,  contrary  to  what  we  assumed. 

Theorem  2:  A  value  c-  \  y  —  x  —  b  is  subsumed  by  an  STP  S  if  and  only  if  d^(S)  =  b 
{Subsumption-Conditioti),  where  d^y(S)  is  the  distance  between  x  andj  in  S. 


21  Let pab  be  a  path  {a,  m,  .  .  nk„b)  in  a  graph  G,  and  let  be  a  node  in  G.  Then pab  .  c  is  the  path  (a,  ni, .  .  nk„h,c)  in  G. 
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Proof:  ”  Suppose  that  the  Subsumption-Condition  holds  for  value  c-  .  By  defini¬ 

tion  of  the  distance  j  —  x  =  d^(S)  holds  in  all  exact  STP  solutions,  so  j  —  x  =  d^(S)  =  b 
holds  in  all  exact  solutions,  and  c-j  holds  in  all  exact  solutions  of  S.  Conversely, 
suppose  the  Subsumption-Condition  does  not  hold,  i.e.  b  <  d^(S)  holds.  Then  there  is 
some  exact  solution  r  of  the  STP  for  which  j  —  x  =  d^^(S)  (as  shown  in  [Dechter,  Meiri 
et  al.  1991]).  Thus,  in  r,  b  <j  —  x,  i.e.  does  not  hold  in  all  of  ^s  exact  solutions. 

Lemma  1:  Adding  more  constraints  to  an  STP  can  only  result  in  monotonic  decrease 
in  the  distances  between  nodes. 

Proof:  Immediate,  by  the  fact  that  adding  a  constraint  to  an  STP  can  only  reduce  the 
solution  set,  and  thus  the  distances  have  to  be  smaller  or  equal  to  the  original  STP. 

Lemma  2:  Adding  a  value  c:{y  —  x  =  b]  to  an  STP  S  when  S  subsumes  q  does  not 
change  the  distance  array  of  S  (i.e.  the  resulting  STP  is  equivalent  to  the  one  before  the 
addition). 

Proof:  Since  c  is  subsumed,  by  Theorem  2,  d^(S)  =  d.  Let  ^  be  a  shortest  path  in  the 
distance  array  from  x  toj/,  and  so  its  weight  is  d^^(S).  Let  us  suppose  now  that  the  dis¬ 
tance  array  of  S  changes  when  c  is  added.  That  means  that  the  new  constraint  c  partici¬ 
pates  in  at  least  one  shortest  path,  let  us  say  on  the  path  from  to  ^  that  before  the 
addition  had  distance  d^^(S).  From  shortest  path  properties  we  get: 

d^S)  =  dJS)  +  dJS)  +  dJS)  =  dJS)  +  b  +  dJS)  (1) 

After  the  addition,  in  the  new  STP  S’  =  S  .  c ,  the  new  distance  d^^(S’)  is: 

d^S’)  =  dJS’)  +  b  +  dJS’)  =  dJS)  +  b  +  dJS)  (2) 

Notice  that  d^^(S’)  —  d^JS)  and  d^^(S’)—  d^^(S)  because  b  is  already  participating  in  the 
shortest  path  from  to  ^  and  therefore  cannot  participate  on  the  shortest  paths  from 
to  J  or  from  x  to  ^  or  a  cycle  would  be  present  on  the  path  from  to  ^  (i.e.  the 
shortest  paths  would  not  be  simple). 

Since  the  distance  array  changed,  the  new  shortest  path  has  to  be  strictly  smaller  the 
one  than  before  <rwas  added  (Lemma  1),  i.e. 

d4S’)<d4S)  (3) 

(1)  and  (2)  imply  that  d^^(S)  =  d^^(S’)  which  contradicts  (3).  Therefore  our  initial  as¬ 
sumption  that  the  distance  array  will  change  is  false. 

Theorem  3:  Let  D=<1/,  C>  be  a  DTP,  let  xl  be  an  assignment  on  D  (i.e.  a  compo¬ 
nent  STP),  and  let  C-  be  a  variable  subsumed  by  A.  Then  xl  is  a  solution  of  D  if  and 
only  if  it  is  a  solution  oi  D’—<Vy  C  —  CA- 
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Proof:  Since  we  assume  that  C-  is  a  subsumed  variable,  then,  there  must  be  a  value  c  :j 
—  x  =  bin  the  domain  of  C  that  is  subsumed  by  A.  Suppose  that  ^  is  a  solution  of  D. 
Then  obviously,  it  is  a  solution  D’—<V,  C  —  C>  since  has  one  less  variable 
(DTP  constraint).  Conversely,  suppose  ^  is  a  solution  of  D\  Since  is  subsumed  by 
A^  it  holds  in  all  of  A^s  exact  solutions,  and  thus  C-  which  is  a  disjunction  involving  <r, 
holds  in  all  of  A's  exact  solutions.  Thus,  if  ^  is  a  solution  of  the  DTP  constraint  C  — 
C- ,  it  also  solves  the  DTP  constraints  C. 

Corollary  1:  Let  ^  be  a  partial  assignment  during  a  DTP  search,  U  be  the  unassigned 
variables,  and  Sub  be  the  set  of  subsumed  variables  in  U.  If  ^  can  be  extended  to  a  so¬ 
lution  over  variables  in  U  —  Sub,  it  can  be  extended  to  a  solution  over  variables  in  U. 
In  other  words,  we  can  remove  the  subsumed  variables  from  the  unassigned  variables 
during  search.  The  solution  to  the  reduced  problem  is  a  solution  to  the  original 

Proof:  By  Theorem  3  if  A’  is  an  extension  of  A  over  the  variables  at  U  —  Sub,  and  A’ 
is  consistent,  then  A’  is  also  a  solution  to  the  original  DTP. 
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1 .  Epilitis(y4,  U,  S)  A  is  the  set  of  assigned  CTP  variables,  U  the  set  of  unassigned  variables,  and  S  a  distanee  and 

predeeessor  array  representing  the  eurrent  STP  */ 

2*  IfU=^  Then 

3*  is  a  solution.  Stop 

4*  Else 

5  *  Let  X  be  a  variable  in  U,  J  i ,  BJ = false 

6.  SB  SBJ=^ 

/*  Set  the  semantie  branehing  justifieation  to  empty  */ 

7.  RSV  If  there  is  value  v.  d(x)  subsumed  by  S  Then 

8.  RSV 

Return  Epilitis(y4  ,  U-{x  },  S) 

9.  RSV  Endlf 

10*  For  eaeh  v  .  d(x)  until  BJ  /*  loop  for  all  values  in  the  eurrent  domain  or  until  the  BJ  flag  is  true  */ 

11* 

A’  ^  A  .  {x  .  V  }  /*  add  value  to  a  (new)  assignment  */ 

12. 

S'  =  maintain-consistency(v,  S) 

13. 

If  is  ineonsistent  Then 

14.  SB 

SBJ  =  justification-value(v,  S’) 

15. 

J  =  J.  SBJ 

16. 

GoTo  36 

17. 

Endif 

18.  FC-off 

If  d(x)  is  singleton  /*  when  FC-off  omit  forward  eheeking  when  d(x)  is  singleton  */ 

19.  FC-off 

20.  FC-off 

Else 

21* 

^be  forward-check(v4 U,  S’) 

22. 

Endif 

23* 

If  ^  — 1  /*  If  we  have  not  reaeh  a  dead  end  . . .  */ 

24* 

Let  J-sons  be  Epilitis(v4  C  U  -  {x},  iS’ )  /*  then  reeursively  eall  Epilitis  */ 

25*CDB 

If  X  .  J-sons  Then 

26* 

J .  J .  J-sons 

27*CDB 

Else  /*  If  the  eurrent  variable  does  not  partieipate  in  the  failure  justifieation  */ 

28*CDB 

J .  J-sons,  BJ=true  /*  then  exit  the  loop  */ 

29*  CDB 

Endif 

30.  SB 

SBJ  =/-sons 

31* 

Else 

32* 

J.  J.  K 

33*NG 

record(project(y4  K),  K) 

34.  SB 

SBJ  =  K 

35* 

Endif 

36.  SB 

If  V  is  not  the  last  value  in  d(x)  /*  remember  SB -constraints  is  the  only  global  variable  */ 

37.  SB 

*S'=maintain-consistency(*S',  reverse(v));  SB-Constraints  =  SB-constraints  .  <re- 

verse(v),  SBJ> 

38.  SB 

If  S  is  ineonsistent 

39.  SB 

un-forward(x),  FinishLoop 

40.  SB 

Endif 

41.  SB 

Endif 

42* 

un-forward(x) 

43*  EndFor 

44*  If  Then 

45* 

For  eaeh  v  .  D{v)  -  d(v)  /*  add  the  justifieations  that  removed  the  values  */ 

/*  from  the  eurrent  domain  */ 

46* 

J.  J.  killers(v) 

47* 

EndFor 

48*NG 

record(project(v4,  J),  J) 

49*  Endif 

50.  SB  Remove  all  semantie  branehing  eonstraints  added  in  this  invoeation  of  Fpilitis  from  SB -Constraints 

51*  RetumJ 

52*  Endif 

Figure  17:  The  Epilitis  Algorithm 
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Appendix  B:  The  Epilitis  Algorithm  in  Detail 

Epilitis,  shown  in  Figure  17,  is  a  generalization  of  the  no-good  recording  algorithm  in 
[Schiex  and  VerfaiUie  1994]  (see  Figure  4  ibid),  and  it  just  adds  code  to  this  algorithm.  The  lines 
common  to  both  algorithms  are  annotated  with  an  asterisk  following  the  Hne  number^^.  EpiHtis  would 
still  correctly  solve  DTP  problems  if  only  these  Hnes  are  included.  The  only  difference  from 
the  plain  no-good  recording  algorithm  would  be  in  forward  checking.  EpiHtis’  forward  check¬ 
ing  mechanism  is  similar  to  the  one  in  Figure  2,  which  takes  into  consideration  the  fact  that  the 
values  of  the  meta-CSP  express  STP-Hke  constraints.  In  other  words,  the  algorithm  in  [Schiex 
and  VerfailHe  1994]  (Figure  4),  with  a  modified  forward-checking  function,  is  a  no-good  re¬ 
cording  DTP  solver. 

Having  said  that,  two  points  must  be  explained  regarding  the  workings  of  EpiHtis:  (i)  the 
additional  ('‘un- starred”)  Hnes  in  the  algorithm  and  (2)  the  exact  way  forward  checking  is  per¬ 
formed.  The  former  is  the  topic  of  the  rest  of  this  section,  and  the  latter  the  topic  of  the  next 
one. 

For  the  rest  of  the  discussion  let  us  assume  that  there  is  avaHable  a  function  forward- 
check(H^  U,  i)  that  given  the  assignment  H,  removes  from  the  variables  U  aH  the  values  in 
their  current  domains  that  are  inconsistent  with  H.  To  check  the  FC-Condition  efficiently  we 
provide  to  the  function  the  distance  array  S  that  corresponds  to  H.  If  a  domain  of  a  variable  is 
reduced  to  the  empty  set,  then  forward-check  should  return  a  justification  K  (also  caHed  the 
mine  Killers  of  the  domain  values;  see  also  [Tsamardinos  2001]  (Section  3.5)  and  [Schiex  and 
VerfaiHie  1994]),  which  is  a  minimal  set  of  variables  in  H  such  that  the  constraints  among 
them  cause  all  the  variables  of  the  domain  to  be  removed. 

The  annotations  SB,  FC-off,  CDB,  NG  and  RSV  shown  next  to  a  Hne  in  the  algorithm 
indicate  which  of  the  corresponding  techniques  (semantic  branching,  FC-off,  confiict  directed 
backjumping,  no-good  recording,  and  removal  of  subsumed  variables,  respectively)  the  Hne 
serves.  For  example,  to  remove  semantic  branching  from  the  algorithm,  we  could  just  remove 
the  Hnes  annotated  with  SB. 

The  Removal  of  Subsumed  Variables  (RSV)  in  Hnes  7-9  is  achieved  by  testing  if,  in  the 
next  variable  to  assign,  there  is  a  value  that  is  subsumed  by  the  current  STP  S.  The  test  can  be 
achieved  by  checking  the  Subsumption-Condition  of  Theorem  2.  If  the  variable  is  subsumed, 
then  it  is  removed  from  the  unassigned  variables  and  EpiHtis  is  recursively  caHed. 

Line  12  propagates  the  value/constraint  of  assignment  {x  .  v)  in  the  current  STP  S’  so 
that  the  distances  of  the  STP  corresponding  to  the  current  assignment  are  avaHable  with  a 
simple  table  lookup.  Recall  that  the  distances  are  required  to  calculate  both  the  FC-Condition 
and  the  Subsumption-Condition.  This  technique  of  maintaining  the  distance  array  was  de¬ 
scribed  in  fuH  in  Section  2.2  and  presented  in  the  algorithm  in  Figure  2. 


22  The  additional  lines  45-47  are  not  in  the  original  paper  [Schiex  and  VerfaiUie  1994].  In  direct  communication  with  the  first 
author  of  the  paper  it  was  established  that  lines  45-47  are  indeed  requited  for  the  algorithm  to  be  complete.  The  experimen¬ 
tal  results  in  that  paper  are  not  invalidated  however  because  lines  45-47  were  included  in  the  implementation  and  were  only 
missing  from  the  pseudo-code  description  of  the  algorithm.  Careful  implementation  of  the  EpUitis  algorithm  also  revealed  a 
typo  in  the  original  publication  of  the  algorithm.  Line  33  appears  originaUy  as  record(project(A,  K),  K)  whUe  it  should  be  re- 
cord(project(A’,  K),  K). 
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When  only  the  basic  forward  checking  DTP  solving  algorithm  is  used,  no  assignment  {x  . 

V  }  will  ever  cause  an  inconsistency  to  the  current  STP  because,  if  it  did  it  would  have  been 
removed  by  forward  checking.  However,  when  semantic  branching  is  used,  it  becomes  neces¬ 
sary  to  check  that  the  constraint  {x  .  ^  }  added  by  semantic  branching  does  not  cause  an  in¬ 

consistency.  This  is  the  reason  for  the  check  at  line  13.  If  we  have  indeed  hit  an  inconsistency 
the  reason  for  it  is  accumulated  in  variable  J  (Hne  15),  which  is  the  justification  to  return  in  case 
of  a  failure  (Hne  50).  Line  14,  annotated  with  SB,  is  explained  in  the  discussion  of  semantic 
branching  below. 

Next  the  algorithm  performs  FC-off  (Hnes  18-20).  Very  simply,  if  there  is  only  one  value  in 
the  current  domain  of  the  current  variable,  we  omit  forward  checking  and  assume  that  it  suc¬ 
ceeded  by  setting  iC  =  -i.  The  ramifications  of  the  omission  were  the  subject  of  Section  6.3 
above.  Otherwise,  we  perform  forward  checking  and  store  in  K  the  mhe  killers  of  the  domain 
that  was  reduced  to  the  empty  set  or  we  store  — i  to  iC  if  there  is  no  such  domain. 

If  we  have  not  hit  a  dead-end,  i.e.  iC=— i  (Hne  23),  then  we  recursively  caH  EpiHtis.  If  it  re¬ 
turns,  then  we  have  failed  to  extend  the  current  assignment  HHo  a  solution  and  the  return 
value  is  a  justification  of  the  faHure,  stored  in  the  variable  J-sons.  If  the  current  variable  partici¬ 
pates  in  this  justification  (Hne  25)  then  we  accumulate  the  justifications  in  variable  J  and  pro¬ 
ceed  (after  Hne  36)  trying  new  values  for  the  current  variable.  If  on  the  other  hand,  the  current 
variable  has  nothing  to  do  with  the  failure,  we  jump  to  Hne  28,  where  BJ  (from  backjumping)  is 
set  to  ^rue  so  that  we  exit  the  loop  and  avoid  trying  any  other  values  of  the  current  variable,  and 
finaHy  return  the  same  reason  J-sons  that  caused  the  failure  in  the  recursive  caH.  On  the  other 
hand,  if  forward  check  fails,  it  returns  the  value  kiHers  K  that  are  accumulated  in  the  overaH 
justification  J  (Hne  32).  Line  33  records  the  no-good  impHed  by  the  dead-end.  Lines  23-35 
(apart  from  the  addition  of  Hnes  30  and  34)  are  exactly  the  same  as  in  the  non-temporal  no¬ 
good  recording  algorithm. 

Perhaps  the  most  compHcated  addition  is  the  Hnes  that  achieve  semantic  branching.  Inte¬ 
grating  SB  with  the  rest  of  the  pruning  techniques  has  impHcations  that  also  affect  the  details 
of  forward  checking  but  for  the  moment  we  restrict  the  discussion  only  to  the  code  that  ap¬ 
pears  in  Figure  17.  When  the  current  assignment  H  .  {x  .  r  }  fails  to  extend  to  a  solution, 
the  code  reaches  Hne  36.  As  already  described  in  detaH  in  Section  3.3,  for  the  rest  of  the  search 
under  assignment  H,  we  can  assume  that  does  not  hold.  Thus,  Hne  36  propagates  the  re¬ 
verse  of  ^in  the  current  STP  S  (i.e.  the  STP  that  corresponds  to  assignment  H).  The  propaga¬ 
tion  might  cause  an  inconsistency  which  would  be  identified  at  Hne  38  in  which  case  there  is 
no  reason  to  try  a  different  value  for  variable  x  and  we  can  exit  the  loop. 

For  reasons  that  we  explained  in  Section  5,  it  is  necessary  to  store  the  semantic  branching 
constraints  that  we  propagate  along  with  the  justification  for  their  addition.  The  store  occurs  at 
Hne  37  where  pairs  <r^  are  stored  in  the  global  variable  SB-Constraints^  where  v  is  an  STP- 
Hke  constraint  and  SBJ  a  justification  (from  semantic  branching  justification).  There  are  three 
different  reasons  why  the  current  value  r  causes  an  inconsistency,  and  correspondingly,  three 
different  Hnes  where  SBJ  is  assigned  a  value.  Value  r  might  directly  cause  an  inconsistency  in 
the  current  assignments!  in  which  case  SBJ  is  the  justification  discovered  by  function  justifi- 
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cation- value  (Figure  4)^^  and  the  assignment  takes  place  at  line  14.  Alternatively,  if  after  as¬ 
signing  value  V  forward  check  failed,  then  SBJ  should  be  the  value  killers  K  that  forward  check 
returned  (Une  34).  Finally,  if  after  assigning  value  v  forward  checked  succeeded  but  the  recur¬ 
sive  call  to  EpiHtis  failed,  then  SBJ  is  assigned  the  value  J-sons  (Une  30).  In  aU  three  cases,  we  fail 
to  extend  ^  .  {x  .  ^  }  to  a  solution  and  we  store  in  SBJ  the  reason  for  the  failure  (i.e.  the 

culprit  of  the  variables  participating  in  the  constraints  that  cause  the  inconsistency). 

The  rest  of  the  EpiUtis  algorithm,  as  already  mentioned,  is  exactly  the  same  as  the  non¬ 
temporal  no-good  recording  algorithm  in  [Schiex  and  Verfaillie  1994]. 

Forward  Checking  and Justifications  in  EpiUtis 

Figure  17  presented  the  EpiUtis  algorithm,  however,  important  details  for  its  implementa¬ 
tion  were  hidden  in  the  forward-check  and  justification-value  functions.  We  now  proceed 
to  the  discussion  of  these  two  functions. 

RecaU  that  we  assumed  that  forward-check(^,  U,  3)  is  a  function  that,  given  the  assign¬ 
ment  A,  removes  from  the  variables  U  aU  the  values  in  their  current  domains  that  are  inconsis¬ 
tent  with  A.  The  STP  S  containing  the  distance  array  that  corresponds  to  ^  is  passed  to  effi¬ 
ciently  check  the  FC-Condition.  A  very  important  feature  of  forward-check  is  that  if  a  do¬ 
main  of  a  variable  is  reduced  to  the  empty  set,  it  should  return  a  justification  K  (also  caUed  the 
m/ue  killers)y  which  is  a  minimal  set  of  variables  in  ^  whose  constraints  cause  the  variables  of 
the  domain  to  be  removed. 

Forward-check  should  check  if  each  remaining  value  v  in  some  current  domain  of  a  vari¬ 
able  should  be  removed  or  not.  A  value  v  should  be  removed,  if,  as  before,  the  FC-Condition 
holds;  is  should  also  be  removed  if  ^  .  {x  .  ^  }  is  a  superset  of  some  recorded  no-good 

<A] />,  i.e.  if  A  .  {x  .  v  }.  That  achieves  forward  checking,  but  it  does  not  solve  the 
problem  of  assembUng  and  returning  a  justification  in  the  case  where  a  variable  domain  is  re¬ 
duced  to  the  empty  set.  Let  us  suppose  that  justification-value(^^  S)  is  responsible  for  return¬ 
ing  the  justification  of  the  removal  of  a  single  value  v  given  the  current  STP  S.  Then,  the  over¬ 
all  justification  for  a  variable  domain  being  empty  is  the  union  of  the  justifications  for  remov¬ 
ing  each  value  originally  in  that  domain. 


Function  justification-value  has  to  be  slightly  modified  from  Figure  4  to  work  for  Epilitis  as  we  will  see  in  the  next  section. 
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forward-check(T,  S,  U) 

1. 

For  each  variable  C  in  U 

2. 

For  each  value  c  :x-y  =  bxy  in  d(C) 

3. 

If  bxy  +  distance  (y,  x,  ^S)  <  0  (FC-Condition)  or 

4. 

A  . 

{C .  c  }  is  a  superset  of  A  \  where  <A  \  no-good- J>  is  a  recorded  no-good 

5. 

Remove  c  from  d(C) 

6. 

lid(C)  =  ^ 

7. 

K=^ 

8. 

For  each  value  v  in  D(C) 

9. 

K  =  K .  justification-value(v,  A,  S) 

10. 

EndFor 

11. 

return  K 

12. 

Endlf 

13. 

Endlf 

14. 

15. 

EndFor 

Return  — i 

Figure  18:  Forward  Checking  for  Epilitis 


Figure  19:  The  function  just-value  for  Epilitis 


Now  we  can  turn  our  attention  to  the  implementation  of  the  function  justification- 
value(^^  ^  which  should  return  a  set  of  variables  from  A,  i.e.  a  justification  that  explains 
why  ^  .  {C  .  v}  cannot  be  extended  to  a  solution  and  thus  why  v  has  to  be  removed  from 
the  current  domain  of  C.  Trivially,  all  the  variables  in  plus  the  variable  C  constitute  a  justifi¬ 
cation  for  the  removal  of  v.  However,  we  can  find  smaller  justifications  that  provide  more  op¬ 
portunities  for  conflict  directed  backjumping  and  search  pruning. 

There  are  two  reasons  why  a  value  v  might  be  removed.  The  first  one  is  if  H  .  {C  .  is 
a  superset  of  where  <A\ /b>  is  a  recorded  no-good,  and  then  the  justification  for  removing 
V  is  The  second  reason  is  if  v  is  removed  because  A  .  {C  .  v]  corresponds  to  an  incon¬ 
sistent  STP  S.  Then,  as  explained  in  detail  in  Section  4,  the  variables  that  cause  the  inconsis¬ 
tency  are  the  ones  that  have  values  assigned  to  them  that  participate  in  a  negative  cycle  in  S. 

As  already  mentioned  in  Section  5,  when  semantic  branching  is  present,  the  current  STP  S 
does  not  directly  correspond  to  the  current  assignment  H,  but  instead  is  formed  by  all  the  con¬ 
straints  in  H  plus  the  semantic  branching  constraints  added.  Thus,  in  Epilitis,  when  the  nega¬ 
tive  cycles  are  identified,  the  corresponding  justification  is  not  just  the  variables  with  values 
that  participate  in  the  cycle,  but  also  the  variables  (i.e.  the  justifications)  that  were  responsible 
for  the  addition  of  the  semantic  branching  constraints  that  participate  in  the  cycle.  These  justi- 

24  See  [Tsamardinos  2001]  for  a  complete  description  of  the  algorithm  for  storing  and  retrieving  no-goods  used  in  Epilitis. 
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fications  can  be  found  in  the  SB-Constraints  structure:  whenever  a  semantic  branching  con¬ 
straint  is  propagated  the  pair  <^,  is  stored  at  SB-Constraints  (line  37,  Figure  17). 

Appendix  C:  FC-Off  Reduces  the  Number  of  Forward-Checks 
but  Increases  Solving  Time 

With  FC-off,  when  the  current  domain  d(C^  of  a  variable  C-  has  been  reduced  to  a  singleton 
set  forward  checking  is  suspended  and  the  constraint  c-jis  assigned  to  C- without  forward 
checking.  FC-off  is  illustrated  in  the  following  example: 

Example  6:  Consider  the  following  DTP: 

C,  :  {c„  :J-X’=5} 

Q:  {c2i  : 

Q  :  \  V  —  x=  5}  .  {c^2  ^0} 

Q-.  =-/0} 

Without  FC-off,  when  the  current  assignment  becomes  —  {C;  .  forward  checking 

will  remove  variable  from  In  the  next  step,  when  the  current  assignment  is  “  {Q 

,  Q  .  is  also  removed  and  thus  d(C^)  becomes  empty  and  the  search  returns  fail¬ 

ure.  In  contrast,  when  FC-off  is  used,  when  the  current  assignment  is  —  {C^  .  forward 
checking  is  suspended  and  nothing  is  removed  from  any  variable’s  domain.  Similarly,  when  the 
current  assignment  becomes  “  {Q  •  ^ii  y  Q  •  C/}?  forward  checking  is  stiU  turned  off, 

and  so  stiU  nothing  is  removed  from  any  variable’s  domain.  Only  when  the  non-singleton  do¬ 
main  d(C2)  is  encountered,  and  the  current  assignment  becomes  =  {Q  .  ,  Q  .  C/  Q 

c^-i)  is  forward  check  called;  at  this  point,  it  will  recognize  the  failure.  Notice  that  when  FC- 
off  is  not  used,  the  algorithm  forward  checks  a  total  of  5+3=8  values  (i.e.,  it  performs  five 
checks  for  A-, — one  for  each  of  the  other  values — removing  one  of  those  values;  and  three 
checks  for  A^.  However,  it  only  expands  two  nodes.  In  contrast,  with  FC-off^  the  algorithm 
checks  2  values  (performed  only  when  the  current  assignment  is  A^  and  it  expanded  three 
nodes. 

The  above  example  shows  that  a  technique  such  as  FC-off  may  or  may  not  increase  the 
performance  of  a  DTP  solving  algorithm.  With  FC-off  there  is  less  forward  checking  but 
more  nodes  are  expanded  (Theorem  17  in  [Stergiou  and  Koubarakis  2000]).  Therefore,  over¬ 
all  effect  of  the  FC-off  technique  wiU  depend  on  the  relative  time  required  to  expand  nodes 
and  to  perform  forward  checking.  There  has  not  been  a  theoretical  analysis  of  the  conditions 
under  which  FC-off  improves  performance,  but  our  experiments,  shown  below,  suggest  that 
FC-off  frequently  degrades  performance  even  though  it  reduces  the  number  of  forward 
checks^^. 

25  .  . 

In  contrast,  Stergiou  and  Koubarakis  note  ‘We  have  also  measured  the  CPU  times  used  by  the  algorithms  we  studied. 
As  expected,  the  CPU  times  are  proportional  to  the  number  of  consistency  checks,”  ([Stergiou  and  Koubarakis  2000],  Sec.  6). 
We  hypothesize  that  this  is  because  in  their  implementation  each  consistency-check  was  not  a  simple  array  lookup.  Instead,  it 
involved  one  constraint  propagation  and  one  constraint  retraction.  Thus,  the  total  time  spent  in  forward-check  greatly  domi¬ 
nates  the  time  spent  in  maintain-consistency. 
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The  arguments  just  given  about  the  flaws  in  using  CC  counts  as  a  metric  of  performance 
are  supported  by  our  experiments,  as  illustrated  in  Table  8  and  Table  9,  which  show  how  well 
different  combinations  of  pruning  strategies  worked  in  Epilitis.  First  consider  Table  8,  which 
shows  the  results  for  N=30.  The  first  column  orders  the  algorithms  in  decreasing  order  ac¬ 
cording  to  the  median  time  taken  in  the  critical  region  where  R=6.  The  second  column  orders 
the  algorithms  by  using  the  median  number  of  CQ,  from  highest  to  lowest.  (So,  in  both  col¬ 
umns,  combinations  that  are  'Tetter”  are  listed  at  the  bottom  on  the  column.)  It  is  easy  to  see 
that  the  median  CCs  favors  the  algorithms  that  use  FC-off  (denoted  with  an  FC  in  their  name) 
and  ranks  them  the  best  while  in  fact  they  are  the  worst  in  terms  of  median  time  performance. 


N=30 

Median-Time 

Median-CCs 

SB-FC 

SB 

CDB-SB-RSV-FC 

CDB-SB 

SB 

SB-RSV 

CDB-SB-RSV 

CDB-SB-RSV 

CDB-SB 

SB-FC 

SB-RSV 

CDB-SB-RSV-FC 

Table  8:  The  ordering  of  performance  for  N=30,  R=6,  from  worst  (top)  to  best  performance. 


N=20 

Median-Time 

Median-CCs 

Nothing 

Nothing 

CDB-RSV-FC 

RSV 

CDB-FC 

CDB 

RSV 

CDB-RSV 

CDB 

SB 

CDB-RSV 

SB-RSV 

CDB-SB-RSV-FC 

CDB-SB 

SB-FC 

CDB-SB-RSV 

SB 

CDB-FC 

SB-RSV 

CDB-RSV-FC 

CDB-SB 

SB-FC 

CDB-SB-RSV 

CDB-SB-RSV-FC 

Table  9:  The  ordering  of  performance  for  N=20,  R=6,  from  worst  (top)  to  best  performance. 

We  repeated  the  same  procedure  as  above  for  N=20,  for  which  a  greater  number  of  ex¬ 
periments  of  pruning  combinations  were  available.  The  results  are  displayed  in  Table  9.  Note 
again  that  there  is  a  large  disparity  between  the  ranking  by  CC  count  and  the  ranking  by  time. 
For  instance,  using  the  CC  metric,  CDB-SB-RSV-FC  is  ranked  the  best,  five  positions  higher 
than  it  reaUy  is  when  we  consider  execution  time. 


53 


References 


[Armando,  Castellini  et  al.  1999]  Armando.,  A.,  C.  Castellini  and  E.  Giunchiglia  (1999).  SAT-based  Proce¬ 
dures  for  Temporal  Reasoning.  5th  European  Conference  on  Planning  (ECP-99). 

[Barber  2QQQ1  Barber,  F.  (2000).  ’’Reasoning  on  Interval  and  Point-based  Disjunctive  Metric  Constraints  in 
Temporal  Contexts/*  Journal  of  Artificial  Intelligence  Research  12:  35-86. 

[Bessiere  1999]  Bessiere,  C.  (1999).  Non-binary  constraints.  Principles  and  Practice  of  Constraint  Pro¬ 
gramming  (CP’99),  Alexandria,  Virginia,  USA,  Springer. 

[Bessiere,  Mesequer  et  al.  1999]  Bessiere,  C.,  P.  Mesequer,  J.  Larrosa  and  E.  Freuder  (1999).  On  forward¬ 
checking  for  non-binary  constraint  satisfaction.  CP'99,  Alexandria,  VA. 

[Bessiere  and  Regin  19971  Bessiere,  C.  and  J.  C.  Regin  (1997).  Arc  consistency  for  general  constraint  net¬ 
works:  preliminary  results.  International  Joint  Conference  on  Artificial  Intelligence  (IJCAr97(), 
Nagoya,  Japan. 

[Chen  and  Beck  2001]  Chen,  X.  and  P.  v.  Beck  (2001).  '’Conflict-Directed  Backiumping  Revisited/*  Jour¬ 
nal  of  Artificial  Intelligence  Research  14:  53-81. 

[Cheng  and  Smith  19951  Cheng,  C.  and  S.  F.  Smith  (1995).  Applying  constraint  satisfaction  techniques  to 
Job-Shop  Scheduling.Technical  Report  CMU-RI-TR-95-Q3  Pittsburgh,  Carnegie  Mellon  Univer¬ 
sity. 

[Cheng  and  Smith  1995]  Cheng,  C.  and  S.  F.  Smith  (1995).  A  constraint  posting  framework  for  scheduling 
under  complex  constraints.  Joint  lEEE/INRIA  conference  on  Emerging  Technologies  for  Factory 
Automation,  Paris,  France. 

[Chleq  19951  Chleq,  N.  (1995).  Efficient  Algorithms  for  Networks  of  Quantitative  Temporal  Constraints. 
Constraints'95. 

[Cormen,  Leiserson  et  al.  19901  Cormen,  T.  H.,  C.  E.  Leiserson  and  R.  L.  Rivest  (1990).  Introduction  to 
Algorithms.  Cambridge,  MA,  MIT  Press. 

[Dechter  19901  Dechter,  R.  (1990).  ’’Enhancement  schemes  for  constraint  processing:  backiumping,  learn¬ 
ing,  and  cutset  decomposition.’*  Artificial  Intellience  41. 

[Dechter  and  Frost  1999]  Dechter,  R.  and  D.  Frost  (1999).  Backtracking  algorithms  for  constraint  satisfac¬ 
tion  problems. Technical  Report,  University  of  California  at  Irvine. 

[Dechter,  Meiri  et  al.  19911  Dechter,  R.,  I.  Meiri  and  J.  Pearl  (1991).  ’Temporal  constraint  networks.’*  Arti¬ 
ficial  Intelligence  49:  61-95. 

[Frost  and  Dechter  19941  Frost,  D.  and  R.  Dechter  (1994).  Dead-end  driven  learning.  National  Conference 
in  Artificial  Intelligence  (AAAI-94). 

[Ginsberg  1993]  Ginsberg,  M.  (1993).  ’’Dynamic  Backtracking.'*  Journal  of  Artificial  Intelligence  Research. 

[Ginsberg  and  McAllester  19941  Ginsberg,  M.  and  D.  McAllester  (1994).  ’’GSAT  and  Dynamic  Backtrack¬ 
ing.’*  Journal  of  Artificial  Intelligence  Research. 

[Gomez,  Selman  et  al.  19981  Gomez,  C.  P.,  B.  Selman  and  H.  Kautz  (1998).  Boosting  combinatorial  search 
through  randomization.  National  Conference  on  Artificial  Intelligence  (AAAr98),  Madison,  Win- 
sonsin. 

[Meiri  1991]  Meiri,  I.  (1991).  Combining  qualitative  and  quantitative  constraints  in  temporal  reasoning. 
National  Conference  in  Artificial  Intelligence  (AAAr91). 

[Mohr  and  Henderson  19861  Mohr,  R.  and  T.  C.  Henderson  (1986).  Arc-consistency  and  path-consistency 
revisited.'*  Artificial  Intelligence  28(225-233). 

[Oddi  and  Cesta  2000]  Oddi,  A.  and  A.  Cesta  (2000).  Incremental  Forward  Checking  for  the  Disiunctive 
Temporal  Problem.  European  Conference  on  Artificial  Intelligence. 

[Pollack  2002]  Pollack,  M.  E.  (2002).  Planning  Technology  for  Intelligent  Cognitive  Orthotics.  6th  Interna¬ 
tional  Conference  on  Al  Planning  and  Scheduling. 

[Pollack  and  Horty  19991  Pollack,  M.  E.  and  J.  F.  Ftorty  (1999).  "There's  More  to  Life  than  Making  Plans: 
Plan  Management  in  Dynamic  Environments."  Al  Magazine. 

[Prosser  1993]  Prosser,  P.  (1993).  "Hybrid  algorithms  for  the  constraint  satisfaction  problem."  Computa¬ 
tional  Intelligence  9. 

[Richards  1998]  Richards,  E.  T.  (1998).  Non-systematic  Search  and  No-good  Leaming.Ph.D.  Thesis  IC 
Parc.  London,  Imperial  College. 


54 


[Roberto  J.  Bayardo  and  Schrag  1977]  Roberto  J.  Bayardo  and  R.  C.  Schrag  (1977).  Using  CSP  Look-Back 
Techniques  to  Solve  Real-World  SAT  Instances.  Fourtheeen  National  Conference  on  Artificial  In¬ 
telligence  (AAAr97). 

[Schiex  and  Verfaillie  19941  Schiex,  T.  and  G.  Verfaillie  (1994).  ’’Nogood  Recording  for  Static  and  Dy¬ 
namic  Constraint  Satisfaction  Problems/*  International  Journal  of  Artificial  Intelligence  Tools 
3(2):  187-200, 

[Schiex  and  Verfaillie  1994]  Schiex.,  T.  and  G.  Verfaillie  (1994).  Stubbornness:  a  possible  enhancement  for 
backiumping  and  no-good  recording.  European  Conference  in  Artificial  Intelligence  (ECAI-94). 

[Staab  19981  Staab,  S.  (1998).  On  Non-Binary  Temporal  Relations.  European  Conference  on  Artificial  In¬ 
telligence. 

[Stergiou  and  Koubarakis  19981  Stergiou,  K.  and  M.  Koubarakis  (1998).  Backtracking  Algorithms  for  Dis¬ 
junctions  of  Temporal  Constraints.  15th  National  Conference  on  Artificial  Intelligence  (AAAI- 
98). 

[Stergiou  and  Koubarakis  2QQQ1  Stergiou,  K.  and  M.  Koubarakis  (2000).  ’’Backtracking  algorithms  for  dis¬ 
junctions  of  temporal  constaints.’*  Artificial  Intelligence  120(1):  81-117. 

[Tsamardinos  19981  Tsamardinos,  I.  (1998).  Reformulating  Temporal  Plans  for  Efficient  Execu- 
tion.Masters  Thesis.  Pittsburgh,  University  of  Pittsburgh. 

[Tsamardinos  2001]  Tsamardinos,  I.  (2001).  Constraint-Based  Temporal  Reasoning  Algorithms  with  Ap¬ 
plications  to  Planning.Ph.D  Thesis  Computer  Science  Department.  Pittsburgh,  University  of  Pitts¬ 
burgh. 

[Tsamardinos,  Vidal  et  al.  20021  Tsamardinos,  I.,  T.  Vidal  and  M.  E.  Pollack  (2002).  ’’CTP:  A  New  Con¬ 
straint-Based  Formalism  for  Conditional,  Temporal  Planning/*  Constraints  (to  appear). 

[Yokoo  1994]  Yokoo,  M.  (1994).  Weak-commitment  search  for  solving  constraint  satisfaction  problems. 
National  Conference  in  Artificial  Intelligence  (AAAI-94). 


55 


APPENDIX  B 


CTP:  A  New  Constraint-Based  Formalism  for  Conditional, 
Temporal  Planning 


loannis  Tsamardinos  (ioannis.tsamardinos@vanderbilt.edu) 

Department  of  Biomedieal  Informaties,  Vanderbilt  University 

Thierry  Vidal  (thierry@enit.fr) 

Produetion  Engineering  Laboratory  (LGP)  -  ENIT,  Franee 

Martha  E.  Pollack  (pollackm@eecs.umich.edu) 

Computer  Seienee  and  Engineering,  University  of  Miehigan 


Abstract.  Temporal  constraints  pose  a  challenge  for  conditional  planning,  because 
it  is  necessary  for  a  conditional  planner  to  determine  whether  a  candidate  plan  will 
satisfy  the  specified  temporal  constraints.  This  can  be  difficult,  because  temporal 
assignments  that  satisfy  the  constraints  associated  with  one  conditional  branch  may 
fail  to  satisfy  the  constraints  along  a  different  branch.  In  this  paper  we  address 
this  challenge  by  developing  the  Conditional  Temporal  Problem  (CTP)  formalism, 
an  extension  of  standard  temporal  constraint-satisfaction  processing  models  used  in 
non-conditional  temporal  planning.  Specifically,  we  augment  temporal  CSP  frame¬ 
works  by  (1)  adding  observation  nodes,  and  (2)  attaching  labels  to  all  nodes  to 
indicate  the  situation (s)  in  which  each  will  be  executed.  Our  extended  framework 
allows  for  the  construction  of  conditional  plans  that  are  guaranteed  to  satisfy  com¬ 
plex  temporal  constraints.  Importantly,  this  can  be  achieved  even  while  allowing  for 
decisions  about  the  precise  timing  of  actions  to  be  postponed  until  execution  time, 
thereby  adding  flexibility  and  making  it  possible  to  dynamically  adapt  the  plan 
in  response  to  the  observations  made  during  execution.  We  also  show  that,  even 
for  plans  without  explicit  quantitative  temporal  constraints,  our  approach  fixes  a 
problem  in  the  earlier  approaches  to  conditional  planning,  which  resulted  in  their 
being  incomplete. 


1.  Introduction 


Classical  planning  (Smith  et  ah,  2000)  assumes  that  a  plan  can  be 
generated  off-line,  prior  to  its  execution,  and  that  execution  consists 
in  the  straightforward  activation  of  the  steps  in  the  plan,  in  an  order 
that  is  consistent  with  the  plan’s  temporal  constraints.  This  assump¬ 
tion  is  satisfied  in  domains  in  which  the  planning  agent  is  omniscient, 
and  thus  knows  at  plan  time  everything  required  about  the  possible 
evolution  of  the  world  during  execution.  In  many  real-world  domains, 
this  assumption  is  violated. 

One  way  to  handle  this  difficulty  is  to  build  the  plan  on-line,  making 
all  decisions  in  a  reactive  fashion.  However,  the  reactive  approach  has  a 
number  of  shortcomings;  in  particular,  when  there  are  real-time  require¬ 
ments  to  be  satisfied,  a  reactive  approach  typically  cannot  guarantee 
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that  they  will  be  met  (nor  can  it  determine  that  they  are  unsatisfiable). 
In  addition,  a  reactive  approach  will  fail  to  take  preparatory  steps  that 
might  be  required  for  the  continuation  of  the  plan  in  unexpected  cir¬ 
cumstances.  Consequently,  an  alternative  approach  has  been  to  develop 
conditional  planning  capabilities  (Peot  and  Smith,  1992;  Pryor  and 
Collins,  1996;  Onder  and  Pollack,  1999).  In  the  conditional  planning 
approach,  plans  are  generated  prior  to  execution,  but  they  include 
both  observation  actions  and  conditional  branches,  which  may  or  may 
not  be  executed,  depending  on  the  execution-time  result  of  certain 
observations. 

In  addition  to  omniscience,  classical  planning  also  assumes  instanta¬ 
neous,  atomic  actions.  However,  modern  planning  systems  (Muscettola 
et  ah,  1998;  Laborie  and  Ghallab,  1995)  indicate  the  growing  need  to 
represent  and  reason  with  metric  temporal  information  and  constraints. 
Temporal  constraints  pose  a  challenge  to  conditional  planning.  In  par¬ 
ticular,  it  is  necessary  for  a  temporal  conditional  planner  to  determine 
whether  a  candidate  plan  can  possibly  satisfy  the  specified  temporal 
constraints.  This  can  be  difficult,  because  temporal  assignments  that 
satisfy  the  constraints  associated  with  one  conditional  branch  may  fail 
to  satisfy  the  constraints  along  a  different  branch. 

In  this  paper  we  address  this  challenge  by  developing  the  Condi¬ 
tional  Temporal  Problem  (CTP)  formalism,  an  extension  of  stan¬ 
dard  temporal  constraint-satisfaction  processing  models  used  in  non¬ 
conditional  temporal  planning.  Specifically,  we  augment  the  previous 
models  by  (1)  adding  observation  nodes,  which  correspond  to  the  time- 
points  at  which  observation  actions  end,  and  (2)  attaching  labels  to  all 
other  nodes  in  the  network.  A  node’s  label  indicates  the  situation(s)  in 
which  the  event  it  denotes  (the  start  or  the  end  of  a  plan’s  action)  will 
be  executed. 

As  we  will  show,  our  extended  framework  allows  for  the  off-line 
construction  of  conditional  plans  that  are  guaranteed  to  satisfy  complex 
temporal  constraints.  Importantly,  this  can  be  achieved  even  while 
allowing  for  the  decisions  about  the  precise  timing  of  actions  to  be 
postponed  until  execution  time,  in  a  least-commitment  manner,  thereby 
adding  flexibility  and  making  it  possible  to  adapt  the  plan  dynamically, 
during  execution,  in  response  to  the  observations  made.  We  also  show 
that,  even  for  plans  without  explicit  quantitative  temporal  constraints, 
our  approach  fixes  a  problem  in  the  earlier  approaches  to  conditional 
planning,  which  resulted  in  their  being  incomplete. 

In  Section  2  we  review  the  temporal  constraint  models  that  inspired 
our  conditional  model,  which  itself  is  depicted  in  Section  3.  Three  levels 
of  consistency  in  conditional  temporal  problems  are  then  defined  in 
Section  4.  Each  is  developed,  in  turn,  in  the  three  subsequent  sections. 


57 


which  provide  algorithms,  discuss  theoretical  complexity  issues,  and 
describe  the  usefulness  of  each  form  of  consistency  to  planning.  In 
particular,  in  Section  8,  we  explain  how  the  use  of  our  approach  in 
conditional  planning  corrects  a  problem  in  the  earlier  approaches.  We 
conclude  this  article  with  a  discussion  of  related  and  future  work. 


2.  Background 

A  variety  of  planning  systems  (Laborie  and  Ghallab,  1995;  Muscettola 
et  ah,  1998),  often  referred  to  as  temporal  planners,  use  an  underlying 
constraint  model  to  query  and  check  the  temporal  aspects  of  a  plan. 
One  of  the  most  popular  models  is  the  Temporal  Constraint  Satisfac¬ 
tion  Problem  (TCSP)  and  its  graph-based  counterpart,  the  Temporal 
Constraint  Network  (TON)  (Dechter  et  ah,  1991).  A  TON  is  a  con¬ 
straint  graph  <  V^E  >  where  the  nodes  V  represent  time-points  (i.e. 
instantaneous  events  associated  with  the  start  or  end  of  actions),  while 
the  edges  E  represent  binary  constraints  on  the  duration  between  two 
time-points  Xi^xj  G  F  of  the  form: 

(/i  <  Xj  —  Xi  <  ui)  y ..  .y  [iji  <  Xj  —  Xi  <  Uji) 

where  /i,  . . .,  . . .,  are  the  lower  and  upper  bounds  of  the 

constraint.  In  other  words  the  time  from  xi  to  Xj  must  lie  in  one  of  the 
intervals  [/i,r^i],  [l2’,U2]’,  •  •  •  [Zn^'^n]-  For  example,  if  x  and  y  represent 
the  start  and  end  time-points  of  action  A  then  the  constraint  (5  < 
y  —  X  <  IC)  y  (20  <  y  —  X  <  25)  specifies  that  the  duration  of  A  is 
between  5  and  10  time  units  (e.g.  minutes)  or  between  20  and  25  time 
units. 

An  interesting  case  is  when  only  one  interval  [Z,  u]  is  allowed  for  each 
edge.  This  is  the  Simple  Temporal  Problem  (STP)  restriction,  whose 
graph  counterpart  is  the  Simple  Temporal  Network  (STN).  Checking 
the  consistency  of  a  TCN  is  known  to  be  NP-complete,  while  in  an 
STN  consistency  can  be  determined  in  polynomial  time,  for  instance 
using  a  local  path-consistency  propagation  algorithm  (Mackworth  and 
Freuder,  1985). 

On  the  other  hand,  TCSPs  may  be  generalized  by  allowing  non¬ 
binary  constraints.  This  is  the  Disjunctive  Temporal  Problem  (DTP) 
(Stergiou  and  Koubarakis,  2000),  which  formally  is  a  constraint-satis¬ 
faction  problem  <  V^E  >  such  that  the  constraints  E  are  arbitrary 
disjunctions  of  STP-like  constraints,  i.e.  each  member  of  E  has  the 
form 

(Zi  <  Xi^  -  Xj^  <Ui)y...y{ln<  Xi^  -  Xj^  <  Un) 
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where  again  /i,  . . DTPs  are  thus  conjunctions  of 
n-ary  disjunctive  constraints:  A^(VjC^j),  where  the  cij  are  of  the  form 
I  <  X  —  y  <  u.  Ks  such,  they  can  represent  any  other  formula  that 
we  can  construct  with  propositions  cij  (e.g.  cn  ^  C12)}  The  most 
frequently  used  way  to  solve  a  DTP  is  to  convert  it  to  a  problem  of 
selecting  one  disjunct,  {1^  <  Xi  —  xj  <  u^)  from  each  disjunctive  con¬ 
straint,  such  that  the  set  of  selected  disjuncts  forms  a  consistent  STP. 
The  DTP  is  consistent  if  and  only  if  at  least  one  of  such  component 
STP  is  consistent.  Checking  the  consistency  of  a  DTP  is  NP-hard, 
and  because  DTPs  include  non-binary  constraints,  it  is  difficult  to 
use  path-consistency  on  them  to  increase  efficiency (Bessiere,  1999). 
However,  several  recent  papers  have  presented  heuristic  techniques  that 
significantly  decrease  the  typical  time  needed  to  check  DTP  consistency 
(Armando  et  ah,  1999;  Stergiou  and  Koubarakis,  2000;  Oddi  and  Cesta, 
2000;  Tsamardinos,  2001). 

Recently,  STPs  have  also  been  extended  to  take  into  account  tem¬ 
poral  uncertainty.  In  STPs  (as  in  TCSPs  and  DTPs),  the  constraint 
between  any  two  time-points  x  and  y  specifies  an  interval  that  is  con¬ 
trollable:  I.e.  the  execution  agent  can  choose  for  {y  —  x)  any  of  the 
values  within  the  allowed  bounds  given  by  the  constraint.  In  realistic 
planning  applications  however,  the  durations  of  some  tasks  or  the  time 
of  occurrence  of  some  events  may  depend  on  external  parameters,  and 
thus  the  actual  value  can  only  be  observed  by  the  agent  at  execution 
time  (e.g.,  (Muscettola  et  ah,  1998)).  Such  contingent  values  may  be 
seen  as  being  assigned  by  Nature  while  the  other  values  are  assigned 
by  the  executing  agent.  A  Simple  Temporal  Problem/Network  with 
Uncertainty  (STPU/STNU)  (Vidal  and  Fargier,  1999)  is  similar  to 
a  STP/STN  except  that  the  edges  are  divided  into  contingent  and 
requirement  edges.  The  finishing  time-point  of  contingent  intervals  are 
controlled  by  Nature  and  hence  observed^  while  others  are  controlled 
and  hence  executed  by  the  agent.  We  will  refer  to  this  model  later  in 
the  paper,  as  there  is  a  strong  relationship  between  consistency  in  an 
STNU  (called  controllability  in  the  STNU  setting)  and  consistency  as 
we  define  it  for  CTPs. 


3.  Conditional  Temporal  Problems 

We  now  introduce  the  Conditional  Temporal  Problem  (CTP)  formal¬ 
ism.  Both  in  this  section  and  throughout  the  remainder  of  the  paper,  we 

^  Note  that  this  is  true  despite  the  fact  that  DTPs  do  not  include  negated  literals, 
because  ^Cij  :  I  <  x  —  y  <  u  can  always  be  rewritten  diS  x  —  y<l\/x  —  y>u, 
approximated  as  close  as  desired  hyx  —  y<l  —  e\/x  —  y>u-\-e. 
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Figure  1.  The  map  for  the  CNLP  plan. 


will  illustrate  our  approach  with  a  simple  example  derived  from  the  one 
used  in  the  original  CNLP  conditional-planning  paper  (Peot  and  Smith, 
1992).  The  example  models  a  plan  for  going  skiing,  either  at  Snowbird 
or  at  Park  City,  starting  from  home,  and  traversing  the  roads  (shown  as 
dashed  lines)  depicted  in  Figure  1.  As  in  the  CNLP  paper,  it  is  possible 
that  the  road  from  point  B  to  Snowbird  and/or  the  road  from  point  C 
to  Park  City  will  be  covered  with  snow  and  impassable.  However,  the 
condition  of  those  roads  can  only  be  observed  from  points  B  and  C, 
respectively.  For  the  purposes  of  the  current  paper,  we  also  introduce 
two  temporal  constraints: 

—  If  the  agent  skis  at  Snowbird,  it  should  arrive  after  1p.m.,  because 
they  offer  great  discounts  in  the  afternoons. 

—  If  the  agent  skis  at  Park  City  instead,  it  should  arrive  at  point  C 
no  later  than  11  a.m.,  because  traffic  is  very  heavy  afterwards. 

In  our  formalism,  we  will  denote  propositions  with  capital  letters 
from  the  beginning  of  the  alphabet,  e.g.  A^B^C.  We  will  denote  literals 
as  A,  or  -lA,  and  conjunctions  of  literals  as  lists  of  literals,  e.g.  AB^C 
denotes  A  A  B  A  ^C. 

Definition  3.1.  A  label  I  is  a  conjunction  of  literals,  e.g.  I  =  ABC. 

Definition  3.2.  Two  labels  li  ,  I2  are  inconsistent  (denoted  as  Inc{li 
,  I2))  iff  h  A  I2  is  False.  E.g.  li  =  AB  and  I2  =  ^B^C  are  inconsistent. 
Two  labels  I1J2  are  consistent  (denoted  Con{lij2))  iff  -'/nc(/l, /2)). 
A  label  li  subsumes  label  l2^  (equivalently,  li  is  more  specific  than 
12)^  iff  h  ^  h-  E-g-  if  h  =  AB^C  and  I2  =  A-iC,  then  li  subsumes  I2 
(denoted  Sub{lij2)) 
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Definition  3.3.  The  set  of  all  possible  labels  defined  with  respect  to 
a  set  of  propositions  P,  is  the  label  universe  of  P,  P*. 

We  are  now  ready  to  formally  define  Conditional  Temporal  Prob¬ 
lems,  inspired  by  the  definitions  of  the  TCSP,  STP  and  DTP. 

Definition  3.4.  A  Conditional  Temporal  Problem  (CTP)  is  a 

tuple  <  V,  P,  P,  OV,  O^P  >,  where: 

P  is  a  finite  set  of  propositions  A,  B,  C,  . . . 

F  is  a  set  of  nodes  (interchangeably  called  variables,  time-points,  events) 

{x,  y,  z,  . . .  } 

P  is  a  set  of  constraints  between  nodes  in  V 

L  :  V  ^  P*  is  a  function  attaching  a  label  to  each  node,  e.g.  L(n)  =  AB 
OV  C  1/  is  the  set  of  observation  nodes 

O  :  P  ^  OV  is  a  bijection  associating  a  proposition  with  an  observation 
node.  0{A)  is  the  node  that  provides  the  truth- value  for  proposition 

A. 


CTPs  can  be  interpreted  as  follows.  When  a  value  is  assigned  to  a 
time  point  in  it  indicates  the  time  of  occurrence  of  that  event.  In 
planning  problems,  the  time  of  occurrence  is  the  time  at  which  an  action 
is  executed.^  The  times  assigned  to  the  nodes  in  a  CTP  must  satisfy 
the  constraints  in  P.  A  major  difference  from  other  non-conditional 
temporal  problems  is  that  a  node  v  G  V  should  only  be  executed 
if  its  label’s  value  becomes  True.  At  the  time  of  their  execution  the 
observation  nodes  provide  the  truth-value  of  propositions,  which  in 
turn  determine  the  truth-value  of  labels.  Note  that  each  proposition 
has  an  implicit  time  point  associated  with  it.  Suppose  that  A  is  a  fluent 
whose  value  may  change  over  time  (for  example  battery-level-low), 
and  further  suppose  that  the  value  of  A  needs  to  be  observed  at  two 
different  times.  Then  the  CTP  will  include  two  distinct  propositions, 
e.g.  and  At2^  each  associated  by  the  function  O  to  a  different 
observation  node.  Therefore  for  each  proposition  in  P  there  exists  a 
unique  observation  node. 

It  is  reasonable  to  assume  that  in  any  well-defined  CTP  there  should 
not  be  any  constraint  relating  nodes  with  inconsistent  labels,  since  those 
nodes  will  never  be  executed  under  the  same  circumstances.  Also,  it 
is  reasonable  to  require  that  when  we  execute  a  node  we  have  to 
know  the  truth- value  of  its  label  L{v).  This  in  turn  implies  that  (i) 
all  nodes  observing  a  proposition  in  L{v)  are  executed  in  all  cases  in 
which  V  is  executed,  and  (ii)  they  are  executed  before  v.  To  ensure 

^  If  the  action  has  temporal  duration,  the  model  includes  two  nodes  for  every 
action:  one  that  denotes  the  time  at  which  execution  of  the  action  begins,  and 
another  that  denotes  the  time  at  which  it  ends. 
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these  requirements  for  any  node  v  for  which  A  appears  in  L{v)^  we 
can  statically  check  that  Sub{L{v)^  L{0{A))  and  add  the  constraint 
0{A)  <  V  to  the  temporal  problem  definition. 

We  now  define  three  types  of  CTPs,  which  differ  in  the  types  of 
constraints  they  allow. 

Definition  3.5.  A  Conditional  Simple  Temporal  Problem  (CSTP) 
is  a  CTP  where  the  constraints  in  E  are  STP-like  constraints,  i.e. 
binary  constraints  (called  edges)  of  the  form  {I  <  y  —  x  <  u).  We  simi¬ 
larly  define  Conditional  Disjunctive  Temporal  Problems  (CDTP)  and 
Conditional  Temporal  Constraint  Satisfaction  Problems  (CTCSP),  by 
analogy  to  DTPs  and  TCSPs. 

Example  3.1.  Figure  2  shows  a  CSTP  that  encodes  part  of  the  ski-trip 
example.  (To  keep  the  example  small,  we  omit  the  part  of  the  plan  that 
involves  traveling  from  point  C  to  Park  City.)  The  vertices  V  represent 
three  types  of  events: 

—  The  start  and  end  points  of  the  go  actions;  (go  x  y)  denotes 
the  action  of  going  from  location  x  to  location  y.  A  node  labeled 
(go  X  y)^',  i.e.  with  subscript  *5,  indicates  the  event  of  starting  the 
action  (go  x  y);  similarly  (go  x  y)^  denotes  the  event  of  ending 
such  an  action  (and  arriving  at  y). 

—  The  observation  events;  (obs  (road  x  y))  denotes  observing  the 
condition  of  the  road  from  x  to  y,  while  located  at  x.  For  clarity  of 
presentation,  we  treat  observation  steps  in  our  examples  as  if  they 
were  instantaneous,  although  in  general  this  is  not  a  requirement. 

—  The  special  Start  event,  which  is  associated  with  a  specific  ar¬ 
bitrary  point  in  time:  in  our  example,  12a.m.  It  is  used  to  encode 
absolute  time  constraints,  for  example,  that  if  the  agent  goes  to 
Snowbird,  he  should  not  arrive  there  until  1p.m.  (13  hours  af¬ 
ter  Start).  In  the  temporal-reasoning  literature,  the  Start  node  is 
usually  denoted  ‘‘TR,”  for  Temporal  Reference  point. 

We  will  follow  the  convention  that  all  non-annotated  edges  in  the  figures 
are  assumed  to  have  bounds  [0,  oo]  and  the  labels  of  the  unlabeled  nodes 
are  True.  There  is  exactly  one  observation  action  (obs  (road  b  s))  that 
provides  the  truth-value  of  A,  namely  whether  the  road  from  6  to  5 
is  open:  thus,  0{A)  =  (obs  (road  b  s)).  Notice  that  this  node  satisfies 
the  two  conditions  identified  above:  It  is  executed  in  every  context  in 
which  steps  labeled  with  A  or  -lA  are  executed  (in  fact,  in  this  example, 
it  is  always  executed),  and  it  is  executed  before  any  of  the  steps  that 
are  labeled  with  A  or  -lA  (note  the  implicit  [0,  oo]  constraint  on  the 
outgoing  arcs  from  it). 
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Figure  2.  The  skiing  plan  as  a  Conditional  Simple  Temporal  Problem.  All  non-an- 
notated  edges  are  assumed  [0,  oo]  and  all  non- annotated  nodes  are  assumed  labeled 
True. 

Definition  3.6.  An  execution  scenario  5  is  a  label  that  partitions 
the  nodes  in  V  into  two  sets  Vi  and  V2  :  Vi  =  {n  ^  V  :  Sub{s^  L{n))} 
and  V2  =  {n  ^  V  :  /nc(5,  L(n))},  i.e.  the  set  of  nodes  that  will  be 
executed  because  s  implies  their  label  is  True^  and  the  set  of  nodes 
that  will  not  be  executed  because  5  implies  their  label  is  False.  An 
execution  scenario  contains  all  the  information  (i.e.  the  value  of  all 
necessary  propositions)  to  decide  which  nodes  to  execute  and  which 
not  to  execute.  We  will  use  SC  to  denote  the  set  of  scenarios  for  a 
given  CTP. 

Theorem  3.1.  Any  complete  truth- assignment  to  the  propositions  in 
P  is  an  execution  scenario  (we  will  call  it  a  complete  scenario^.  (See 
Appendix  for  proof). 

Definition  3.7.  A  scenario  projection  of  the  CTP  <  L, 

OF,  O,  P  >  of  an  execution  scenario  5,  denoted  as  Pr(s),  is  the 
temporal  non-conditional  problem  <  Vi,Pi  >,  where  Fi  =  {n  G  F  : 
Sub{s^  L{n))}  and  Ei  =  {{vi.,V2)  ^  E  :  vi^V2  G  Fi}.  This  will  be  an 
STP,  DTP,  or  TCSP  depending  on  the  constraints  in  E.  Put  simply, 
the  scenario  projection  of  an  execution  scenario  5  is  the  set  of  nodes  of 
the  CTP  that  will  be  executed  under  5  and  all  the  constraints  among 
them. 

Example  3.2.  In  our  skiing  example,  there  are  only  two  possible  exe¬ 
cution  scenarios:  A  and  -lA,  where  Pr(A)  includes,  intuitively,  all  the 
nodes  that  are  executed  when  the  road  to  Snowbird  is  clear  and  the 
agent  skis  there,  and,  similarly,  Pr(-iA)  includes  all  the  nodes  that  are 
executed  when  the  road  to  Snowbird  is  closed  and  the  agent  instead 
heads  towards  Park  City  via  point  C.  For  a  more  interesting  example, 
consider  the  CSTP  in  Figure  3,  which  has  nodes  {TP,  y,  2;,  re,  'y. 
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AB 


Figure  3.  The  CTP  of  the  Example  3.2. 

q}  with  labels  {True^  A,  -lA,  AB^  ^AC^  -lA-iC},  respectively. 

Suppose  that  TR  =  0{A)  observes  A^  y  =  0{B)  observes  B^  and 
2;  =  0{C)  observes  C.  The  CSTP  thus  has  the  typical  structure  of  a 
conditional  plan  in  which  TR  is  initially  executed  and  A  is  observed;  if  it 
is  true  y  is  executed,  otherwise  2;  is  executed;  and  so  on.  The  execution 
scenarios  AB^  ABC  and  AB^C  all  refer  to  the  same  execution,  i.e. 
under  each  of  them,  the  nodes  TR^y^  and  re,  but  no  other  nodes,  will 
be  executed. 

Definition  3.8.  Two  execution  scenarios  are  equivalent  execution 
scenarios  if  they  induce  the  same  partition  on  the  set  of  nodes.  The 
‘‘equivalent  execution  scenarios”  relation  induces  an  equivalence  class 
relation  R.  A  class  in  R  contains  all  scenarios  that  are  equivalent. 

Definition  3.9.  A  scenario  is  a  minimum  execution  scenario  if  it 

contains  the  minimum  number  of  propositions  compared  to  all  other 
scenarios  in  its  “equivalent  execution  scenario”  class. 

In  Example  3.2  above,  scenarios  ABC^  AB^C^  and  AB  all  belong  to 
the  same  equivalence  class,  with  AB  being  the  minimum  execution  sce¬ 
nario  of  the  class.  For  this  example,  there  are  four  minimum  scenarios: 
{AR, 
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4.  Notions  of  Consistency 


CTPs  have  a  different  notion  of  consistency  than  their  non-conditional 
counterparts  TCSPs,  DTPs,  STPs,  and  most  other  temporal  reason¬ 
ing  problems.  In  the  conditional  case,  consistency  cannot  be  defined 
simply  as  the  existence  of  an  assignment  to  the  time-points  (nodes) 
that  satisfies  the  constraints.  This  interpretation  fails  to  take  account 
of  the  fact  that  in  the  CTP  modeling  a  temporal  plan,  propositions  are 
usually  only  observed  at  execution  time.  We  thus  present  three  notions 
of  consistency,  which  differ  from  one  another  in  the  assumptions  made 
about  when  observation  information  is  known. 

In  the  first  case,  we  require  a  notion  of  consistency  that  allows  for 
off-line  scheduling,  i.e.  allows  precise  times  for  all  events  to  be  deter¬ 
mined  before  execution  begins.  Here  it  is  reasonable  to  assume  that 
the  scheduling  algorithm  has  no  information  about  the  outcome  of  the 
observations.  Therefore  it  should  schedule  the  nodes  in  such  a  way  that 
the  constraints  are  satisfied  no  matter  how  the  observations  turn  out. 
If  such  a  schedule  exists,  then  we  will  say  that  the  CTP  is  Strongly 
Consistent. 

As  a  second  case,  consider  an  agent  that  plans  for  a  number  of 
initial  future  states.  Each  initial  state  corresponds  to  a  set  of  initial 
truth-values  of  some  propositions,  i.e.  an  execution  scenario.  The  spe¬ 
cific  scenario  is  unknown  at  planning  time,  but  it  will  be  known  to 
the  execution  agent  prior  to  execution.  Then,  it  is  necessary  for  the 
planning  agent  to  verify  that  no  matter  which  initial  state  turns  out 
True^  the  execution  agent  has  a  way  to  execute  the  plan.  If  this  is 
possible,  the  CTP  is  Weakly  Consistent. 

The  third,  most  complicated  and  typical  case,  assumes  that  in¬ 
formation  about  the  outcome  of  observations  becomes  known  during 
execution.  Unlike  the  first  case,  however,  it  allows  the  decisions  about 
the  timing  of  events  to  be  made  dynamically  at  execution  time.  We 
will  call  a  CTP  Dynamically  Consistent  if  it  can  be  executed  so  that 
no  matter  what  the  outcome  is  for  the  upcoming  observations,  the 
current  partial  solution  (i.e.  the  assignment  of  values  to  time-points) 
can  be  extended  so  that  all  constraints  are  satisfied. 

These  notions  of  consistency  are  similar  to  those  developed  for  ST- 
PUs,  where  consistency  is  defined  in  terms  of  controllability  (Vidal  and 
Fargier,  1999).  Essentially,  a  network  is  controllable  if  there  is  a  strategy 
for  executing  the  timepoints  under  the  agent’s  control  that  satisfies 
all  requirements.  Three  primary  levels  of  controllability  had  also  been 
identified.  In  Strong  Controllability^  there  is  a  static  control  strategy 
that  is  guaranteed  to  work  in  all  situations.  In  Weak  Controllability^ 
for  all  situations  there  is  a  ‘‘clairvoyant”  strategy  that  works  if  all 


65 


uncertain  durations  are  known  when  the  network  is  executed.  And  in 
Dynamic  Controllability^  it  is  assumed  that  each  uncertain  duration 
becomes  known  (is  observed)  just  after  it  has  finished,  and  it  requires 
that  an  execution  strategy  depend  only  on  the  past  outcomes.  Strong 
and  Dynamic  Controllability  have  been  shown  to  be  tractable  (Vidal 
and  Fargier,  1999;  Morris  et  ah,  2001),  while  Weak  Controllability  is 
conjectured  to  be  co-NP-complete  (Vidal  and  Fargier,  1999). 

Definition  4.1.  A  schedule  T  of  a  CTP  <  V,  L,  OV,  O,  P  >  is  a 
mapping  V  ^  Ji,  i.e.  a  time  assignment  to  the  nodes  in  V.  We  denote 
with  T(v)  the  time  assigned  to  node  v. 

Definition  4.2.  An  execution  strategy  St  is  a  function  from  the  set 
of  scenarios  for  a  CTP  to  a  schedule  St  :  SC  T.  A  viable  execution 
strategy  is  one  such  that  St{s)  is  a  solution  to  the  projection  Pr(s) 
for  each  scenario  5  G  SC. 

Definition  4.3.  Given  a  scenario  5  and  a  schedule  T,  for  each  node  x  in 
the  projection  scenario  of  5,  we  can  determine  the  set  of  all  observations 
performed  before  time  T(x),  along  with  their  outcomes.  We  will  call 
the  set  of  the  observation  outcomes  before  time  T{x)  the  observation 
history  of  x  relative  to  scenario  s  and  schedule  T  and  will  denote  it 
as  H{x^  5,T). 

Definition  4.4.  A  CTP  <  V,  P,  L,  OV,  O,  P  >  is  Weakly  Consistent 

if  there  exists  a  viable  execution  strategy  for  it,  i.e.  every  projection 
Pr(s)  is  consistent  in  the  STP/DTP/TCSP  sense. 

Definition  4.5.  A  CTP  <  V,  P,  P,  OV,  (9,P  >  is  Strongly  Consis¬ 
tent  if  there  is  viable  execution  strategy  St  such  that,  for  every  pair 
of  scenarios  si  and  52,  and  variable  x  executed  in  both  scenarios, 

[S'i(si)](a;)  =  [S'i(s2)](a;) 

Thus,  an  execution  strategy  that  satisfies  the  definition  of  Strong 
Consistency  assigns  a  fixed  time  to  each  executable  timepoint  irre¬ 
spective  of  the  observation  outcomes.  The  idea  behind  finding  a  single 
schedule  is  similar  to  that  of  finding  a  conformant  plan  (Goldman  and 
Boddy,  1996). 

Definition  4.6.  A  CTP  <  V,  P,  P,  OV,  O,  P  >  is  Dynamically 
Consistent  if  there  is  a  viable  execution  strategy  St  such  that,  for 
every  variable  x  and  pair  of  scenarios  si  and  52, 

Con{s2^  H{x^  5i,  *9^(51))  V  Con(5i,  P(x,  52,  St{s2))) 

[S't(si)](a:)  =  [S't(s2)](a;) 
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In  the  definition  above^,  Con{s2^  H{x^  si^  St{si)))  means  that  the 
set  of  observation  outcomes  uncovered  before  x  in  scenario  si  forms  a 
label  that  is  still  consistent  with  scenario  52  at  the  time  at  which  x  is 
to  be  performed  in  si.  Thus,  at  time  point  x,  the  agent  has  not  yet 
distinguished  between  scenarios  si  and  52.  Therefore  it  must  assign 
to  X  the  same  time  in  si  and  52.  The  same  arises  in  the  opposite 
case  {x  in  scenario  52  while  si  is  still  feasible).  An  execution  strategy 
that  satisfies  the  above  definition  ensures  that  the  scheduling  decisions 
that  are  taken  (while  executing)  only  use  information  available  from 
previous  observations. 

To  compare  the  three  notions  of  consistency,  reconsider  the  CSTP  of 
Figure  2  as  defined  in  Example  3.1.  Is  the  network  Strongly  Consistent? 
We  can  immediately  see  that,  if  A  is  True  (i.e.  the  agent  observes  that 
the  road  to  Snowbird  is  open),  then  it  must  arrive  and  then  imme¬ 
diately  leave  point  b  no  sooner  than  noon,  i.e.  (go  home  b)^;  >  12, 
given  the  constraints  13  <  (go  b  s)^;  —  Start  <  00,  0  <  (go  b  s)^'  — 
(go  home  b)^;  <  0,  and  1  <  (go  b  s)^;  —  (go  b  s)^'  <  1.  That  is,  because 
the  agent  must  not  arrive  in  Snowbird  before  1p.m.,  there  is  no  place 
to  wait  at  point  6,  and  it  takes  an  hour  to  get  from  b  to  Snowbird,  it 
must  arrive  at  b  no  earlier  than  noon.  An  analogous  argument  let  us 
deduce  that  if  A  is  False  it  must  arrive  at  b  no  later  than  10a.m.,  i.e. 
(go  home  b)^;  <  10  Thus,  there  is  no  way  to  construct  a  schedule  for 
this  network  without  knowing  the  truth- value  of  A.  Hence,  the  CTP  is 
not  Strongly  Consistent. 

However,  it  is  Weakly  Consistent.  To  prove  this  we  just  have  to 
provide  a  consistent  schedule  for  each  scenario.  In  this  example  there 
are  two  execution  scenarios  si  =  A  and  52  =  “'A,  which  means  only  the 

^  We  point  out  that  our  consistency  definitions  and  algorithms  would  still  be  valid 
had  we  defined  labels  as  any  propositional  formula  on  P* .  We  chose  to  restrict  labels 
to  conjunctions  for  illustration  purposes.  Also,  note  that  although  they  are  sym¬ 
metric,  both  disjuncts  Con{s2,  H{x,si,  St{si))  and  Con{si,  H{x,  S2,  St{s2)))  are 
required  for  the  definition  to  cover  only  the  set  of  plans  that  are  actually  executable. 
Here  is  an  example  that  shows  that  using  only  one  of  the  disjuncts  is  not  enough. 
Consider  the  CTP  with  nodes  x,y,z,w,  L{z)=A^  L{w)=^A,  L{x)=L{y)=True,  and 
0{A)  =  y^  and  constraints  x  —  y  ^  [—5,  5],  z  —  y  =  5,  w  —  y  =  15,  z  —  x  =  10,  and 
w  —  X  =  10.  11 A  is  True,  only  the  schedule  Ti{x)  =  0,  Ti{y)  =  5,  and  Ti{z)  =  10  and 
all  of  its  translations  Ti  are  consistent.  If  A  is  False,  only  the  schedule  T2{y)  =  0, 
T2{x)  =  5,  and  T2{w)  =  15  and  all  of  its  translations  are  consistent.  Ti  and  T2  define 
the  execution  strategy  St  with  St{A)  =  Ti  and  St{-^A)  =  T2.  The  CTP  is  obviously 
not  dynamically  consistent  since  we  need  to  know  the  value  of  A  before  execution  in 
order  to  decide  whether  we  should  follow  Ti  or  T2,  but  A  is  known  only  after  x  has 
been  executed  in  Ti.  However,  H{x,^A,T2)  =  ^A  and  so  ^Con{A,  H{x,^A,T2)). 
If  only  this  disjunct  was  used  in  the  definition,  the  antecedent  of  the  implication 
would  be  False,  and  the  constraint  always  satisfied  and  thus  St  would  satisfy  the 
definition,  falsely  implying  the  CTP  is  dynamically  consistent. 
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truth-value  of  A  discriminates  between  possible  scenarios.  One  consis¬ 
tent  schedule  for  which  we  will  call  Ti,  assigns  (go  home  b)^'  a  time 
of  10  a.m.  (with  all  the  other  time  points  being  directly  derivable  from 
that,  e.g.,  (go  home  b)^;  is  assigned  noon).  A  consistent  schedule  for 
52,  T2  assigns  (go  home  b)^'  a  time  of  8  a.m.  The  execution  strategy 
St,  where  St(si)  =  Ti  and  St{s2)  =  T2  is  viable.  So,  provided  only 
that  the  value  for  A  is  known  before  execution  starts,  the  agent  simply 
needs  to  pick  the  corresponding  schedule  Ti  or  T2. 

Is  the  network  Dynamically  Consistent?  In  the  discussion  above  we 
showed  that  T((go  home  b)^;)  must  be  greater  than  or  equal  to  12  if 
A  is  observed  True,  and  less  than  or  equal  to  10  if  A  is  observed  to 
be  False.  In  turn,  this  forces  T((go  home  b)^')  to  be  greater  than  or 
equal  to  10  if  A  is  True,  and  less  than  or  equal  to  8  if  A  is  False.  If 
we  could  observe  A  before  starting  out  on  the  journey,  i.e.  before  event 
(go  home  b)^',  then  we  could  distinguish  between  the  two  scenarios, 
determine  which  one  we  fall  into,  and  schedule  our  departure  from 
home  accordingly.  However,  the  problem  is  set  up  such  that  being  at 
point  6  is  a  precondition  for  the  observation  action.  Thus,  there  is  no 
way  to  perform  the  observation,  and  determine  the  value  of  A,  ‘dn  time” 
to  schedule  the  departure.  The  example  is  not  dynamically  consistent. 

Let  us  now  state  an  obvious  property  of  the  three  notions  of  consis¬ 
tency,  that  is  similar  to  the  corresponding  one  in  STPU: 

Theorem  4.1.  Strong  Consistency  ^  Dynamic  Consistency  ^  Weak 
Consistency 


5.  Strong  Consistency 

We  now  present  an  important  property  of  Strong  Consistency  for  CTPs. 

Theorem  5.1.  A  CTP  <  V,  L,  E,0V,0,  P  >  is  Strongly  Consistent 
if  and  only  if  the  (non-conditional)  temporal  problem  <  V,E  >  is 
consistent.  (See  Appendix  for  proof). 

The  implication  of  the  above  theorem  is  that  we  can  perform  Strong 
Consistency  checking  by  using  specialized  algorithms  for  non-condi¬ 
tional  temporal  problems  such  as  IDCP  (Chleq,  1995)  for  STPs,  known 
techniques  (Schwalb  and  Dechter,  1997)  for  TCSPs,  and  Epilitis  (Tsa- 
mardinos,  2001)  for  DTPs. 
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5.1.  Uses  of  the  Strong  Consistency  Concept  for  Planning 


A  plan  that  is  represented  as  a  strongly  consistent  CTP  can  be  executed 
according  to  a  fixed  schedule,  in  the  sense  that  every  action  it  includes 
has  an  assigned,  specific  time.  Not  all  of  the  actions  will  occur  in  every 
scenario.  Thus,  the  plan  is  contingent  in  the  same  sense  as  plans  gener¬ 
ated  by  CNLP  (Peot  and  Smith,  1992):  Certain  actions  may  or  may  not 
be  executed,  depending  on  the  results  of  execution-time  observations. 
A  contingent  plan  with  a  fixed  schedule  should  be  contrasted  with 
temporally  contingent  plan,  in  which  the  times  at  which  actions  are 
performed  also  depends  upon  such  observations.  Plans  with  necessary 
temporal  contingencies  will  be  dynamically  consistent  (as  discussed  in 
Section  7)  but  not  strongly  consistent. 

Thus,  Strong  Consistency  is  a  restrictive  type  of  consistency,  mean¬ 
ing  that  a  CTP  might  not  be  Strongly  Consistent  but  nonetheless  can 
be  executed.  Nevertheless,  since  we  can  employ  existing  algorithms  and 
systems  for  determining  Strong  Consistency  it  is  possible  to  build  a 
temporal  and  conditional  planner  using  those  systems.  Such  a  planner 
performs  a  search  in  the  plan  space  adding  appropriate  actions,  obser¬ 
vations  and  temporal  constraints  to  resolve  the  conflicts  (threats)  in  the 
CNLP  style.  The  consistency  of  the  underlying  temporal  constraints  in 
the  CTP  representing  the  current  plan  can  then  be  determined. 


6.  Weak  Consistency 
6.1.  Weak  Consistency  Checking 

It  is  easy  to  design  a  brute  force  algorithm  for  checking  Weak  Consis¬ 
tency.  The  task  involves  finding  a  solution  to  the  temporal  subproblem 
Pr(si)  for  every  execution  scenario  Si.  Thus,  the  two  steps  of  the 
algorithm  are: 

1.  Find  the  set  of  execution  scenarios  SC. 

2.  Check  the  consistency  of  the  non-conditional  problem  Pr(s),V5  G 

SC. 

Let  us  examine  a  specific  example  of  the  above,  on  the  CSTP  of  Fig¬ 
ure  2.  The  two  projections  Pr(A)  and  Pr(-iA)  and  all  the  constraints 
in  these  two  projections  are  shown  in  Figure  4.  Consistency  can  be 
easily  proven  in  each  projection. 

We  can  improve  on  this  algorithm  by  noticing  that  Pr(si)  =  Pr(sj) 
for  every  two  equivalent  scenarios  Si  and  sj.  Thus,  we  only  need  to 
select  one  scenario  Si  from  each  class  in  R  of  Definition  3.8  and  check 
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[2,  2] 


[13,8] 


Figure  4-  The  projected  STPs  of  Figure  2  for  all  scenarios.  Pr(-iA)  is  shown  in  the 
top  part  and  Pr(A)  in  the  bottom  part. 


the  consistency  of  Pr(si).  It  might  even  be  desirable  to  select  the  min¬ 
imum  execution  scenario  as  the  representative  of  its  class.  Then  the 
first  step  of  the  algorithm  is  the  problem  of  finding  the  set  of  minimum 
execution  scenarios.  We  solve  this  problem  in  (Tsamardinos,  2001) 
where  we  present  an  algorithm  that  calculates  this  set  without  explicitly 
enumerating  all  possible  scenarios.  We  also  prove  the  complexity  result 
that  Weak  CSTP  Consistency  is  co-NP-Complete  (see  Theorem  A.l  in 
the  Appendix). 

Another  way  to  improve  the  Weak  Consistency  checking  algorithm 
is  to  perform  the  second  step  of  the  algorithm  incrementally.  For  exam¬ 
ple,  when  solving  the  sequence  of  problems  Pr(si),  Pr(s2), . . .  there  is 
often  shared  computation  between  problems  Pr(si)  and  Pr(si+i).  The 
order  of  consideration  of  each  Pr(s)  highly  influences  the  amount  of 
computation  that  can  be  shared.  An  algorithm  that  employs  this  idea 
and  calculates  an  appropriate  order  of  incremental  consistent  checks  for 
the  series  of  Pr(si)  is  again  presented  in  (Tsamardinos,  2001)  (Chapter 
6)  for  the  CSTP  case. 

6.2.  Uses  of  the  Weak  Consistency  Concept  for  Planning 

If  a  plan  is  represented  as  a  weakly  consistent  CTP  that  is  not  also 
dynamically  or  strongly  consistent,  this  means  that  there  is  always  a 
non- contingent  plan  for  any  set  of  observations,  but  to  execute  that 
plan,  we  must  know  the  observations  at  the  start  of  execution. 

We  now  suggest  a  possible  use  of  Weak  Consistency  for  planning 
purposes,  namely  in  planning  architectures  that  are  based  on  plan¬ 
merging.  Examples  of  such  architectures  are  Workflow  Management 
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Systems  (Georgakopoulos  et  al.,  1995),  PRS  (Myers,  1997),  the  Plan 
Management  Agent  (Tsamardinos  et  ah,  2000),  and  Autominder  (Pol¬ 
lack  et  ah,  2002)  to  name  a  few.  In  these  systems,  there  is  a  library 
of  plan  (or  workflow)  schemata,  and  whenever  a  new  goal  arrives,  a 
plan  from  the  plan  library  is  selected  and  subsequently  merged  with 
the  system’s  existing  commitment  structure,  i.e.  the  set  of  (partially) 
instantiated  plans  which  it  has  already  adopted.  The  plan  to  be  merged 
in  the  context  will  depend  both  upon  the  new  goal  to  satisfy  and  the 
current  set  of  commitments.  The  conditions  set  by  the  latter  form 
a  ‘‘scenario”  for  which  there  should  exist  a  corresponding  schedule, 
which  makes  Weak  Consistency  relevant.  The  plan  library  may  include 
a  number  of  different  schemata  that  achieve  a  given  goal  G.  A  CTP 
can  be  used  to  compactly  represent  all  such  schemata.  For  example,  if 
Pi  achieves  goal  G  when  A  is  True^  and  plan  P2  achieves  G  when  -lA, 
then  we  can  build  a  CTP  that  simultaneously  represents  both  Pi  and 
P2  by  attaching  appropriate  labels  A  and  -lA  on  the  temporal  variables 
in  the  plans.  Now  assume  that  Pi  and  P2  share  a  large  part  of  their 
structure  and  differ  only  in  a  few  preparatory  steps.  It  is  easy  to  see 
that  the  CTP  representation  can  attach  the  label  True  to  all  common 
steps  and  this  way  both  display  the  shared  structure  and  remove  the 
redundancy. 

As  a  simple  example  consider  the  CTP  shown  in  Figure  5(a),  which 
encodes  the  four  non-conditional  plans  in  Figure  5(b).  (Imagine  that 
label  A  denotes  “fuel  tank  empty”  and  label  B  denotes  “load  over  2000 
lbs”).  Not  only  is  the  CTP  encoding  more  compact,  but  checking  its 
consistency  using  efficient  Weak  Consistent  checking  algorithms  will  be 
faster,  by  definition,  than  individually  checking  the  consistency  of  each 
the  non-conditional  temporal  plans. 


7.  Dynamic  Consistency 
7.1.  Dynamic  Consistency  Checking 

We  now  turn  to  Dynamic  Consistency.  In  order  to  distinguish  between 
the  execution  of  the  same  node  in  different  scenario  projections,  we  use 
N{x^s)  to  denote  node  x  in  Pr{s).  Also,  let  us  denote  with  Diff ^^{si) 
the  set  {N{v^si)}  of  all  nodes  v  in  si  that  provide  observations  with 
outcomes  that  differ  from  the  corresponding  ones  in  52. 

To  prove  Dynamic  Consistency  of  a  CTP  we  need  to  identify  a  viable 
execution  strategy  St  satisfying  the  conditions  of  Definition  4.6.  The 
condition  Gon{s2^  H{x^  5i,  St{si)))  is  satisfied  if  and  only  if  at  the  time 
7V(x,  si)  is  executed,  there  is  no  observation  node  N{v^  si)  in  Diff ^^{si) 
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B 


(A) 


(d) 


(B) 

Figure  5.  (A)  A  CTP  encoding  four  different  plan  schemata.  (B)  The  non-condi¬ 
tional  plan  schemata  represented  by  the  CTP  in  (A). 

that  has  been  executed  yet;  otherwise,  we  could  distinguish  between 
the  two  scenarios.  This  directly  translates  to  the  following  equation  for 
Con{s2,H{x,  si,St{si)))\ 

Con{s2^H{x^si^St{si)))  ^  l\  N{x^si)  <  N{v^si) 

and  similarly  for  Con{si^  H{x^  S2^  St{s2))-  Thus,  we  can  rewritte  the 
condition  in  Definition  4.6  as: 

{  A  Nix,si)<Niv,si)} 

V{  A  Ar(rE,52)  <iV(^;,52)} 

Niv,S2)eDiff,^is2) 

^N{x,si)  =  N{x,S2)  (1) 

On  the  left-hand  side  of  the  implication,  we  have  simply  replaced  each 
of  the  two  Con  conditions  from  Dehnition  4.6  with  a  conjunction  over 
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DynConsistency(CTP  Ctp) 

1.  DTP  D:<V,C  >=<  UiEi  >, 

where  Pr{si)  =<  Vi^Ei  >  are  the  projected  scenarios. 

2.  For  each  pair  of  scenarios  52 

3.  For  each  node  v  that  appears  in  both  52 

4.  C  =  C  ADC{v,si,S2) 

5.  EndFor 

6.  EndFor 

7.  If  D  is  consistent,  return  Dynamic-Consistent 

8.  Else  return  non-Dynamic-Consistent 


Figure  6.  The  Dynamic  Consistency  algorithm 


times  of  observation  nodes;  the  right-hand  side  of  the  implication  has 
remained  unchanged. 

The  main  idea  behind  the  consistency  checking  algorithm  is  to  view 
the  above  condition  of  the  definition  as  a  (disjunctive)  constraint  be¬ 
tween  nodes  N{x^s):  These  together  with  the  set  of  all  nodes  N{x^s) 
for  every  node  x  and  scenario  5  of  the  original  CTP  define  a  new 
temporal  problem,  namely  a  DTP  D.  With  this  reformulation,  an  exe¬ 
cution  strategy  St  defines  a  schedule  T  of  D  and  vice  versa  by  setting 
[S't(5)](x)  =  T{N{x^  s)).  The  aim  is  to  add  appropriate  constraints 
to  D  so  that  a  solution  schedule  of  D  will  correspond  to  a  Dynamic 
execution  strategy  and  vice  versa.  So,  in  addition  to  the  constraints 
resulting  from  Equation  (1),  we  need  to  impose  on  the  nodes  of  D  all 
the  constraints  in  every  projection  Pr(s).  Then,  every  solution  to  D 
will  satisfy  both  the  constraints  in  each  projection  (thereby  guarantee¬ 
ing  that  the  corresponding  strategy  will  be  viable),  and  the  Dynamic 
Consistency  conditions.  These  ideas  lead  to  the  design  of  the  algorithm 
in  Figure  6,  where  DC(x,  81,82)  (called  a  DC  constraint)  is  used  as  a 
shorthand  of  Equation  (1)  above. 

Let  us  trace  the  algorithm  on  a  specific  example  such  as  the  CSTP  of 
Figure  2  where  0{A)  =  (obs  (road  b  s)).  Line  1  of  the  algorithm  creates 
a  DTP  with  all  the  nodes  and  edges  in  the  two  projections  Pr(A) 
and  Pr(-iA).  The  result  is  shown  in  Figure  7  (with  some  additional 
constraints  explained  below).  To  simplify  the  equations  we  renamed 
the  nodes  (go  home  b)^',  (go  home  b)^;,  (obs  (road  b  s),  (go  b  s)^',  and 
(go  b  s)^;  as  x^y^z^w^  and  v.  We  notice  that  Diff ^^{A)  =  {N{z^A)} 
and  Diff j^{-^A)  =  {N{z^^A)}  and  so  the  DC  for  Starts  DC{Start^  A, 
^A),  is 


N {Start,  A)  <  N{z,  A)  M  N{Start,^A)  <  N{z,A) 
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[13,8] 


Figure  1.  The  DTP  created  by  the  Dynamic  Consistency  algorithm,  including 
Pr(-iA)  (top  part),  Pr(A)  (bottom  part),  and  DC  constraints  between  them. 


^  N {Starts  A)  =  N {Starts  ^A). 

Since  Start  is  before  2;  in  both  cases,  N {Starts  A)  =  N {Starts  ^ A). 
Similarly,  we  find  that  7V(x,  A)  =  7V(x,  -lA),  N{y^  A)  =  N{y^  ^A)^  and 
N{z^A)  =  N{z^^A).  For  nodes  w  and  v  the  antecedent  of  the  DC 
implication  is  always  False  (they  occur  after  2;  in  both  scenarios)  and 
so  the  DC  constraint  is  already  satisfied.  The  result  after  adding  all  the 
DC  constraints  in  Line  4  of  the  algorithm  is  shown  in  Figure  7.  The 
resulting  DTP  is  actually  an  STP  and  it  is  inconsistent,  indicating  that 
the  original  CTP  is  not  Dynamically  Consistent. 

Suppose  instead  that  the  constraints  ordering  2;  after  y  are  dropped 
and  2;  can  now  be  executed  any  time  after  Start.  In  particular,  other 
constraints  permitting,  it  could  be  scheduled  before  x  and  y.  Then,  the 
DC  constraint  for  x  specifies  that  either  x  occurs  before  2;  in  which  case 
N{x^A)  =  N{x^^A)^  or  it  occurs  after  2;  in  both  scenarios.  Thus,  the 
DC  constraints  are  disjunctive  in  general  (recall  that  a  ^  bis  equivalent 
to  -la  V  6).  In  case  where  2;  is  allowed  to  be  executed  before  x  and  y  the 
CTP  is  Dynamically  Consistent.  Semantically  this  corresponds  to  the 
case  where  we  observe  whether  the  road  from  6  to  5  is  open  before  we 
leave  home.  In  that  case,  we  can  decide  when  to  start  the  trip  for  each 
different  scenario. 

Notice  that  in  Equation  (1)  if  the  observation  nodes  in  all  scenarios 
are  constrained  to  be  ordered  with  respect  to  each  other,  then  the 
conjunctions  over  all  N{v^Si)  G  Diff ^.{si)  can  be  substituted  with  the 
single  minimum  of  the  order.  Then  Equation  (1)  becomes 

N{x,si)  <  N{n,si)\/ N{x,S2)  <  N{m,S2)  ^  N{x,si)  =  N{x,S2)  (2) 
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AB 


Figure  8.  A  CTP  with  two  observation  nodes  unordered  with  respect  to  each  other. 


n  and  m  being  the  nodes  for  which  7V(n,  5i),  7V(m,  52)  are  minimum 
in  the  order  of  the  nodes  in  Diff^^{si)  and  respectively. 

In  general,  observation  nodes  that  are  constrained  to  be  scheduled 
after  others  in  the  sets  Diffg^{si)  and  are  ruled  out  of  the 

conjunctions  in  Equation  (1). 

A  more  complicated  example  is  shown  in  Figure  8  where  two  ob¬ 
servation  nodes  0{A)  =  x  and  0{B)  =  y  are  unordered  with  each 
other  so  Equation  (1)  cannot  be  simplified  to  the  form  of  Equation 
(2).  Let  us  consider  node  x  and  scenarios  si  =  AB  and  52  =  A^B. 
Then,  Diff,^{si)  =  {7V(y,5i)}  while  Biff, ^{32)  =  {N{y,S2)}.  Thus, 
DC{x^  5i,  S2)  is  the  constraint  7V(x,  5i)  <  N{y^  32)  V  N(x,  32)  <  N{y^  32) 
^  N{x^3i)  =  7V(x,52).  If  we  decide  to  perform  the  observation  for  A 
first,  i.e.  X  <  y^  then  DC{x^  3i^  32)  becomes  N{x^3i)  =  7V(x,52).  The 
resulting  STP  is  shown  in  Figure  9(a).  In  the  other  case  (where  we 
defer  the  observation  of  A)  we  end  up  with  the  STP  in  Figure  9(b) 
where  there  is  no  constraint  between  N{x^3i)  and  7V(x,52).  Since  the 
observations  are  unordered,  the  DC  constraints  are  disjunctive  and 
represent  in  a  DTP  both  of  these  alternative  STPs  of  (a)  and  (b). 
The  original  CTP  is  Dynamically  Consistent,  if  and  only  if  one  of  these 
alternatives  is  consistent. 

It  is  important  to  note  that  we  reduced  the  consistency  checking 
problem  to  a  DTP  because  DTPs  can  represent  n-ary  disjunctive  con¬ 
straints.  TCSPs  and  STPs  do  not  allow  this,  and  thus  would  not  sat¬ 
isfy  our  requirements.  Constraints  of  this  type  are  typical  of  Dynamic 
Consistency  in  CTPs  and  make  the  problem  intractable  in  general. 
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Figure  9.  The  STP  projections  with  DC  constraints  for  each  order  of  observations. 
Directed  edges  are  assumed  [0,  oo]  and  undirected  edges  denote  [0,  0]  constraints. 


This  contrasts  with  Dynamic  Controllability  in  STPUs  in  which  con¬ 
straints  can  be  reduced  to  simple  STP  constraints,  hence  allowing  the 
design  of  polynomial-time  solution  algorithms.  Thus,  DTP  solving  al¬ 
gorithms  such  as  Epilitis  (Tsamardinos,  2001),  which  include  a  number 
of  highly  effective  heuristic  pruning  techniques,  will  have  a  direct  effect 
on  Dynamic  Consistency  checking  in  CTPs. 

Regarding  the  complexity  of  the  DTP  D,  notice  that  D  contains 
(9(|E|  \SC\)  variables,  where  V  is  the  set  of  variables  in  the  CTP  and 
SC  the  set  of  (minimal)  scenarios  whose  number  is  in  the  worst  case 
exponential  to  the  number  of  propositions  \P\.  In  the  worst  case  the 
constraints  are  disjunctive  and,  when  put  in  Conjunctive  Normal  Form, 
may  create  an  exponential  number  of  disjunctive  clauses.  Nevertheless, 
some  structural  properties  of  the  CTP  help  in  reducing  the  complexity. 

First,  as  we  have  noted  above,  when  the  observation  nodes  are  or¬ 
dered  with  respect  to  each  other  in  every  scenario,  the  DC  constraints 
are  given  by  Equation  (2)  which  is  a  great  simplification  over  Equation 
(1).  Additionally,  if  each  node  is  ordered  with  respect  to  every  obser- 
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vation  node  in  all  scenarios,  then  the  antecedent  of  each  DC  constraint 
can  be  statically  checked.  In  this  case,  the  DC  constraint  either  becomes 
7V(x,  si)  =  7V(x,  S2)  or  it  is  already  satisfied.  For  instance  a  CSTP  can 
then  be  made  equivalent  to  a  larger  STP,  since  no  disjunctive  con¬ 
straints  are  added  to  the  problem,  which  allows  very  efficient  Dynamic 
Consistency  checking. 

7.2.  Uses  of  Dynamic  Consistency  Concept  for  Planning 

Dynamic  Consistency  checking  can  be  used  to  build  a  temporal  and 
conditional  planner.  It  is  easy  to  see  that  by  appropriately  modifying 
the  CNLP  algorithm  (Peot  and  Smith,  1992)  it  is  possible  to  allow  the 
simultaneous  representation  of  and  reasoning  with  quantitative  tempo¬ 
ral  constraints  and  conditional  branches.  When  a  temporal  constraint 
X  <  y  is  added  to  the  CTP  representing  the  conditional  plan  (e.g.  to 
resolve  a  conflict),  the  Dynamic  Consistency  algorithm  can  determine 
whether  the  resulting  plan  is  executable.  Moreover,  this  notion  of  ‘‘ex¬ 
ecutable”  goes  beyond  that  of  traditional  planning  system,  because  it 
allows  for  observations  to  be  made  at  execution  time  in  plans  in  which 
timing  constraints  depend  on  observation  outcomes.  Dynamic  Consis¬ 
tency  checking  can  also  support  the  merging  of  such  richly  expressive 
plans  at  execution  time,  e.g.  to  handle  new  goals  that  arise  during 
execution  (Tsamardinos,  2001)  (Chapter  7). 

In  order  to  execute  a  Dynamically  Consistent  plan  we  can  instead 
execute  the  DTP  D  to  which  we  reduced  the  problem.  Notice  that  we 
should  execute  only  one  node  N{x^Si)  for  every  scenario  Si  since  they 
semantically  correspond  to  the  same  event  and  the  same  CTP  node. 
Of  course,  the  algorithm  guarantees  that  N{x^Si)  =  N{x^Sj)  in  all 
appropriate  cases  and  avoids  confusion.  We  can  identify  at  least  three 
ways  D  can  be  executed:  (i)  We  compute  a  solution  to  D  and  execute 
that.  This  is  the  least  flexible  approach  since  it  commits  to  a  specific 
schedule  (solution)  of  D.  (ii)  We  find  and  flexibly  execute  a  consistent 
component  STP  of  D.  Consistent  components  STPs  are  returned  by 
DTP  solvers  such  as  Epilitis  (Tsamardinos,  2001)  and  can  be  flexibly 
executed  with  algorithms  such  as  in  (Tsamardinos  et  ah,  1998).  (iii) 
We  flexibly  execute  the  DTP  directly,  retaining  all  possible  scheduling 
flexibility,  using  the  algorithm  in  (Tsamardinos  et  ah,  2001). 

Finally,  we  note  that  because  typical  conditional  plans  satisfy  both 
of  the  conditions  mentioned  at  the  end  of  the  previous  subsection,  the 
performance  of  the  DC  algorithm  during  plan  construction  and  merging 
is  likely  to  be  higher  than  in  the  general  case.  In  addition,  conditional 
planners  generate  plans  where  the  number  of  distinct  execution  sce¬ 
narios  is  linear  in  the  number  of  propositions.  We  suspect  that  in 
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this  case  the  Dynamic  Consistency  algorithm  we  presented  is  actually 
polynomial  in  the  number  of  original  CTP  variables  and  propositions 
(Tsamardinos,  2001)  (Chapter  6).  We  intend  to  formalize  these  ideas  on 
performance  improvements  in  our  future  work. 


8.  Improved  Conditional  Planning 

In  the  previous  section,  we  noted  that  by  using  a  Dynamic  Consistency 
algorithm,  one  can  extend  traditional  conditional  planners  to  support 
quantitative  temporal  constraints.  It  is  essential  to  manage  those  con¬ 
straints;  if  they  are  ignored,  then  the  planner  risks  generating  incorrect 
plans.  For  instance,  a  CNLP-style  planner  would  generate  a  conditional 
plan  for  our  skiing  example  if  it  simply  ignored  the  two  additional 
temporal  constraints  (either  arrive  at  Snowbird  after  1  p.m.,  or  else 
arrive  at  point  C  on  the  way  to  Park  City  before  11  a.m.).  But  such 
a  plan  would  be  useless,  because,  as  we  have  already  seen,  given  the 
temporal  constraints  the  plan  is  dynamically  inconsistent  and  there  is 
no  way  of  executing  it. 

Of  course,  the  traditional  conditional  planners  (Peot  and  Smith, 
1992;  Pryor  and  Collins,  1996;  Onder  and  Pollack,  1999)  were  not 
designed  to  deal  with  quantitative  temporal  constraints.  But  they  do 
perform  a  limited  form  of  temporal  reasoning,  in  order  to  deal  with 
ordering  constraints,  and  it  turns  out  that  even  for  plans  with  only 
ordering  constraints,  there  are  clear  advantages  to  using  the  dynamic 
consistency  approach. 

CNLP  propagates  context  information  only  along  causal  links  and 
conditioning  links,  but  not  along  ordering  constraints.  We  assume  that 
this  choice  was  made  so  that  if  a  step  x  with  context  True  is  promoted 
after  a  step  y  of  context  A,  then  the  context  of  x  remains  True  and  so  x 
can  be  reused  to  provide  causal  links  to  steps  in  other  contexts,  thereby 
potentially  reducing  the  amount  of  planning  required  and  resulting  in 
smaller  plans. 

Nevertheless,  this  method  might  reject  valid  (i.e.  executable)  plans. 
An  example  is  shown  in  Figure  10(a).  The  bold  edges  correspond  to 
causal  links,  and  the  lighter  edges  denote  ordering  constraints  added 
by  threat  resolution.  We  suppose  that  step  v  clobbers  both  the  causal 
link  t  ^  u  and  the  causal  link  x  ^  y;  hence  v  has  been  promoted  and 
demoted  respectively  to  resolve  the  conficts.  We  further  suppose  that 
2;  clobbers  the  latter  causal  link  and  has  been  promoted  after  y. 

When  CNLP  checks  whether  an  ordering  constraint  exists  between 
a  pair  of  nodes  5  and  2;,  it  essentially  computes  the  transitive  closure 
and  determines  whether  5  <  2;  holds.  Since  the  context  information  is 
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A  A 


fA  ^ 

(a) 


A  A 


fA  ^ 

(b) 


Figure  10.  (a)  A  CNLP  plan  whose  handling  by  CNLP  reasoning  falsely  induces 
that  s  is  necessarily  before  (b)  The  two  projections  of  the  plan:  ^  is  allowed 
before  s  in  both. 


ignored  in  this  calculation,  CNLP  essentially  calculates  Strong  Consis¬ 
tency  of  the  induced  CTP.  However,  in  Figure  10(b)  the  two  projections 
of  the  plan  are  shown,  and  it  is  easy  to  see  that  5  and  2;  are  actually 
unordered  with  respect  to  each  other. 

As  already  mentioned  in  Section  5,  Strong  Consistency  is  a  restric¬ 
tive  type  of  consistency  and  plans  that  are  executable  (i.e.  Dynamically 
Consistent)  might  not  be  Strongly  Consistent.  Thus,  CNLP  is  not  com¬ 
plete  and  it  might  reject  valid  plans^  unlike  what  is  conjectured  in  the 
original  CNLP  paper.  In  the  above  example,  if  2;  is  ordered  before  5 
and  this  is  the  only  valid  plan,  CNLP  will  reject  it  as  inconsistent  even 
though  it  is  Dynamically  Consistent  (assuming  A  is  observed  before 
this  portion  of  the  plan). 
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9.  Related  Approaches  and  Conclusions 


As  far  as  we  know  only  two  approaches  might  be  compared  to  our 
work.  The  first  paper  by  Schwalb  et  al.  (Schwalb  et  ah,  1994)  separates 
propositional  and  temporal  reasoning,  addressing  expressive  proposi¬ 
tional  and  temporal  constraints,  to  process  deduction  and  hypothetical 
reasoning  on  knowledge  bases.  The  authors  define  a  ’’Conditional  Tem¬ 
poral  Network”  model,  in  which  some  constraints  are  dependent  on  a 
condition  and  are  only  used  if  that  condition  is  True.  The  aim  is  to  make 
queries  in  the  base  such  as  ”is  formula  F  consistent  with  the  current 
constraints?” .  Although  such  a  model  may  be  seen  as  a  general  logical 
framework  for  doing  conditional  temporal  reasoning,  it  is  insufficient 
for  our  purpose  for  two  reasons.  First,  the  approach  is  static  and  does 
not  deal  with  the  dynamic  aspects  of  plan  execution:  the  time  at  which 
a  condition  is  known  to  be  True  or  False  is  not  consider,  which  for 
planning  purposes  is  crucial.  Second,  unlike  Weak  Consistency,  which 
determines  whether  all  scenarios  are  consistent,  they  determine  whether 
there  is  at  least  one  consistent  scenario.  This  is  sufficient  when  process¬ 
ing  queries  on  a  knowledge  base,  when  one  interpretation  is  searched 
for,  but  in  our  planning  context  that  would  only  mean  there  exists  one 
unique  scenario  in  which  the  plan  will  not  fail. 

The  second  and  more  interesting  paper  is  that  of  Barber  (Barber, 
2000),  which  combines  quantitative  temporal  constraints  and  alterna¬ 
tive  contexts  in  a  kind  of  networks  that  we  will  call  BarN^.  Barber 
defines  a  temporal  problem  where  constraints  (instead  of  nodes)  are 
annotated  with  a  label  (in  his  terminology  a  context).  Consistency  in 
a  BarN  corresponds  to  Weak  Consistency  in  a  CTP. 

A  primary  difference  between  BarNs  and  CTPs  is  thus  that  the 
former  is  based  on  conditional  constraints  while  the  latter  is  based  on 
conditional  events.  In  the  Appendix  (Theorem  A. 2)  we  show  that  we 
can  use  the  latter  to  represent  the  former  and  thus  our  formalism  is  at 
least  as  general  as  Barber’s.  Because  contexts  in  BarNs  are  associated 
with  labels,  not  nodes,  the  translation  from  conditional  planning  is  not 
as  clear  as  with  CTPs,  which  very  naturally  associate  an  observation 
with  a  node  and  attach  appropriate  labels  to  subsequent  nodes.  CTPs 
can  then  readily  check  various  forms  of  consistency,  using  the  techniques 
described  earlier.  In  contrast,  with  a  BarN  representation,  the  planner 
has  to  construct  the  context  hierarchy  itself  given  the  observations. 
Perhaps  the  most  important  ramification  of  this  implicit  treatment 
of  observations  is  that  the  notion  of  Dynamic  Consistency  cannot  be 


^  BarN  denotes  Barber  Networks. 
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defined  for  BarNs.  This  is  because  the  truth  value  of  the  contexts  is 
not  associated  with  a  particular  time-point. 

Unlike  BarNs,  our  new  Conditional  Temporal  Problem  formalism 
is  geared  towards  planning  and  execution  purposes.  It  is  a  constraint- 
based  formalism  for  temporal  reasoning  in  the  face  of  uncertain-or 
contingent-events,  and  we  have  described  its  usefulness  for  conditional 
planning.  There  are  many  avenues  for  future  research,  of  which  we 
highlight  a  few.  CTPs  deal  with  temporal  uncertainty  arising  from  the 
outcome  of  observations,  while  STPUs  handle  uncertainty  regarding 
the  timing  of  uncontrollable  events.  Obviously,  a  hybrid  model  and 
algorithms  that  handle  both  sources  of  uncertainty  would  be  highly 
desirable.  We  are  also  working  on  identifying  minimal  structural  re¬ 
quirements  for  CTPs  that  will  enable  polynomial-time  Dynamic  Con¬ 
sistency  algorithms.  In  parallel,  we  are  also  investigating  efficient  Weak 
Consistency  algorithms. 
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Appendix 

Theorem  3.1.  Any  complete  assignment  to  the  propositions  in  P  is 
an  execution  scenario. 

Proof.  Let  5  be  a  complete  truth-assignment  to  the  propositions  in  P,  I 
be  a  label  and  pi  . .  .pn  be  the  propositions  that  appear  in  I  (either 
positive  or  negated).  The  set  of  pi  will  also  appear  in  5  since  5  is 
complete.  If  any  pi  appears  with  different  sign  in  5  and  I  then  5  and 
I  are  inconsistent.  Otherwise,  if  all  pi  appear  with  the  same  sign  in  5 
and  I  then  5  subsumes  1.  So,  every  node,  no  matter  what  its  label  is, 
will  either  belong  to  Vi  or  V2  in  Definition  3.6.  □ 

Theorem  5.1.  A  CTP  <  E^OV^O^  P  >  is  Strongly  Consistent 
if  and  only  if  the  (non- conditional)  temporal  problem  <  V^E  >  is 
consistent. 

Proof.  The  theorem  states  that  we  can  determine  Strong  Consistency 
by  ignoring  the  label  and  the  observation  information  of  the  nodes  in 
the  CTP  and  just  calculate  consistency  as  we  would  for  an  STP,  TCSP, 
or  DTP  depending  on  the  kind  of  constraints  in  E. 

‘‘4=”  Suppose  that  the  CTP  is  Strongly  Consistent.  Then  we  will 
show  that  the  temporal  problem  <  F,  P  >  is  also  consistent.  Let  T  be 
a  schedule  of  all  nodes,  such  that  T{x)  =  [S't(5^)](x),  where  si  some 
scenario  where  x  is  executed.  T{x)  is  a  function  because  (i)  x  appears 
in  at  least  one  scenario  (or  it  can  be  removed  from  the  CTP),  and  (ii)  a 
Strong  execution  strategy  specifies  a  unique  value  to  every  [S't(5^)](x) 
for  all  scenarios  Si  where  x  appears.  Because  St  is  viable,  T  satisfies 
the  constraints  in  all  Pr(si)  =<  Vi^Ei  >,  for  every  scenario  Si.  Since 
T  satisfies  the  constraints  in  every  set  Ei  it  satisfies  the  constraints  in 
their  union  =  P.  Thus,  T  is  a  solution  to  <  F,  P  >  and  so  the 
latter  is  consistent. 

Suppose  that  the  temporal  problem  <  F,  P  >  is  consistent;  we 
will  prove  that  the  CTP  <  F,  P,  P,  OF,  O,  P  >  is  Strongly  Consistent. 


83 


Let  T  be  any  solution  oi  <V^E  >  (it  has  to  have  at  least  one  since  it 
is  consistent).  For  every  5^,  T  is  also  a  solution  of  Pr(si)  =<  Vi^Ei  > 
(ignoring  any  irrelevant  assignments  T{x)  where  x  does  not  appear  in 
Vi)  since  Ei  C  E.  The  execution  strategy  St{si)  =  T  is  viable  (since 
T  is  a  solution  to  every  Pr(si)  =<  Vi^Ei  >)  and  also  [S't(5^)](x)  = 
[St{sj)]{x)  =  T{x)^\/x  as  the  definition  of  Strong  Consistency  requires. 

□ 

Theorem  A.l.  Weak  CSTP  Consistency  checking  is  co-NP- complete. 

Proof.  We  will  prove  the  result  by  translating  in  polynomial  time  and 
space  a  SAT  problem  to  the  co-problem  of  checking  Weak  Consistency, 
the  co-problem  being  finding  a  scenario  Si  such  that  Pr(si)  is  incon¬ 
sistent.  Specifically,  we  will  create  a  CSTP  given  a  SAT  problem  such 
that  the  SAT  problem  has  a  solution,  if  and  only  if  there  is  a  scenario 
Si  such  that  Pr(si)  is  inconsistent. 

Given  the  SAT  problem  with  Boolean  variables  B  =  {x, . . . ,  y}  and 
clauses  of  the  form  =  (x  V  . . .  V  y  V  -12; . . .  -■re),  i  =  1 . . .  K  we  create 
a  CSTP  <  F,  E^  L,  OTV,  O,  P  >  as  follows:  The  set  of  propositions  is 
P  =  B  =  {x, . . .  ,  y}.  For  each  clause  Ci  =  {xW . .  . .  .W^w)  and 

each  variable  appearance  x  or  ^x  in  Ci  we  create  a  time-point  X  that 
we  include  in  F,  with  label  L{X)  =  x  or  L{X)  =  ^x  respectively.  Let 
us  denote  with  Clause{Ci)  the  nodes  of  the  CSTP  that  were  included 
because  of  Ci  .  Since  we  are  checking  for  Weak  Consistency  it  does  not 
matter  which  nodes  are  observation  nodes.  The  last  thing  to  define  are 
the  constraints  between  the  nodes.  There  is  an  constraint  between  a 
variable  X  G  Clause{Ci)  to  each  variable  Y  with  consistent  label  in 
Clause{C(^iJ^l^^r^o(ix)'•  Y  —  X  =  —1  (we  will  drop  the  modK  clause  in 
the  rest  of  the  proof  for  clarity;  just  remember  that  the  nodes  in  the 
last  Clause  (Ci)  are  connected  to  the  nodes  in  the  first  Clause{Ci)). 
The  translation  is  obviously  linear  in  the  number  of  SAT  variables  and 
linear  in  the  number  of  clauses  of  the  SAT  problem. 

Figure  11  illustrates  the  proof  concept.  It  presents  an  example,  by 
showing  the  resulting  CSTP  from  the  SAT  problem  {x  y  y  y  z)  A  {x  y 
^y  y  z)  A  (-ix  V  -ly  V  ^z)  A  (-ly  V  z).  The  labels  of  each  node  appear 
on  its  top  right  corner.  Notice  that  there  are  three  propositions  in  the 
CSTP  for  the  three  SAT  variables  x^y^z  that  appear  in  the  labels, 
either  as  positive  or  negative  literals,  and  eleven  CSTP  nodes  one  for 
each  appearance  of  a  variable  in  any  clause.  The  nodes  in  the  CSTP 
are  arranged  in  columns  corresponding  to  Clause{Ci)C  =  1...4  and 
are  named  with  the  variable  of  the  corresponding  proposition  and  the 
index  i  .  The  edges  are  connected  from  a  node  in  Clause{Ci)  to  all  the 
nodes  in  Claus e{CiJ^i).  The  order  of  appearance  of  clauses  in  the  SAT 
problem  is  arbitrary.  Also  recall  that  all  the  edges  from  a  node  X  to 
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Figure  11.  The  CSTP  resulting  from  translating  a  simple  example  TSAT  problem. 


a  node  Y  correspond  to  the  constraint  Y  —  X  =  —1  not  shown  in  the 
figure  for  clarity.  Notice  also  that  there  are  no  edges  between  nodes 
with  inconsistent  labels,  e.g.  x  and  ^x. 

Let  us  assume  that  the  SAT  problem  has  a  solution  {x  =  True^  . . 
y  =  True^  z  =  False.,. . re  =  False}.  Since  this  solution  makes  True  at 
least  one  variable  in  a  clause,  it  will  make  True  the  label  of  at  least  one 
CSTP  time-point  within  Clause{Ci)  .  We  will  call  a  time-point  whose 
label  becomes  True  in  Clause (Ci)  Li  (there  may  be  more  than  one). 
Notice  that  each  Li  has  to  have  an  edge  to  because  it  cannot  be 
the  case  that  Li  has  a  label  x  and  a  label  ^x  by  the  way  the 

SAT  solution  is  constructed  (it  never  assigned  x  both  True  and  False 
at  the  same  time).  Thus  the  set  {L^,  i  =  1 . . .  K}  forms  a  negative  cycle 
(with  weight  (—1)  x  K).  There  must  be  at  least  one  scenario  s  that 
makes  all  the  nodes  in  {LiC  =  I . . .  K}  True.  Namely,  let  5  be  the 
complete  scenario  that  corresponds  to  the  SAT  solution  {s  is  a  scenario 
by  Theorem  3.1).  In  other  words,  Pr(s)  is  an  inconsistent  projected 
STP.  In  the  above  example,  the  SAT  solution  {x  =  True^  y  =  False ^  z  = 
True}  makes  the  labels  of  the  CSTP  nodes  xi^X2^y2^y^^yA^  Z2  ,and 
2^4  True  and  all  the  rest  False.  The  former  set  forms  at  least  one  negative 
cycle,  e.g.  {xi^X2^y3^  Z4^}.  This  completes  the  proof  that  if  the  SAT 
problem  has  a  solution,  the  CSTP  is  not  Weakly  Consistent. 

We  will  now  prove  the  converse,  namely  that  if  the  SAT  problem  has 
no  solution,  then  the  CSTP  is  Weakly  Consistent.  Take  any  complete 
assignment  to  the  SAT  variables.  Any  such  assignment  also  corresponds 
to  a  scenario  of  the  CSTP  (by  Theorem  3.1).  If  SAT  has  no  solution, 
for  every  such  assignment /complete  scenario  5  there  is  at  least  one 
clause  Ci  that  is  not  True^  or  in  other  words,  all  the  SAT  literals 
of  Ci  have  to  be  False  too.  Thus,  all  the  time-points  of  Clause  (Ci) 
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are  inconsistent  (not  executed)  under  scenario  5.  But,  by  the  way  the 
CSTP  is  constructed,  every  cycle  (negative  or  not)  has  to  go  through 
all  clauses.  Since  no  time-point  in  Clause{Ci)  becomes  True  under  5  , 
there  can  be  no  negative  cycle  in  Pr(s)  for  any  scenario  s. 

The  above  argument  shows  that  checking  Weak  CSTP  Consistency 
is  co-NP-hard.  Since  checking  if  an  STP  is  consistent  is  a  polynomial 
problem,  co-Weak  Consistency  is  also  in  co-NP  and  thus  the  problem 
is  co-NP-complete. 

□ 


Theorem  A. 2.  Every  (conditional)  constraint  of  the  form  li<xi  — 
yi  <  ui  y  ...  y  Ik  <  Xk  —  Vk  ^  Uk  that  should  hold  only  when  label  I  is 
True,  can  he  represented  with  only  conditional  events. 

Proof.  For  any  constraint  that  we  want  to  represent  of  the  form  li  < 
x—y  <  V/2  <  s—t  <  U2  with  condition  (label)  /,  we  create  the  dummy 

nodes  w,  z,  u,  v  all  having  label  1.  We  then  insert  constraints  requiring 
that  the  pairs  of  time-points  {x,w},  {y^z},  {5,1^},  and  {t,v}  co-occur 
(e.g.  0  <  X  —  re  <  0),  and  we  also  add  the  (unconditional)  constraint 
h  <  w  —  z  <  ui  y  I2  <  u  —  V  <  U2.  This  way  when  I  is  True,  nodes 
w,z,u  and  v  will  be  executed  and,  because  they  co-occur,  the  original 
(conditional)  constraint  on  nodes  x,y,s  and  t  will  be  imposed.  □ 
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Abstract 

The  aging  of  the  world’s  population  poses  a  challenge  and 
an  opportunity  for  the  design  of  intelligent  technology.  This 
paper  focuses  on  one  type  of  assistive  technology,  cogni¬ 
tive  orthotics,  which  can  help  people  adapt  to  cognitive  de¬ 
clines  and  continue  satisfactory  performance  of  routine  ac¬ 
tivities,  thereby  potentially  enabling  them  to  remain  in  their 
own  homes  longer.  Existing  cognitive  orthotics  mainly  pro¬ 
vide  alarms  for  prescribed  activities  at  fixed  times  that  are 
specified  in  advance.  In  contrast,  we  describe  Automin¬ 
der,  a  system  we  have  designed  that  uses  AI  planning  and 
plan  management  technology  to  carefully  model  an  individ¬ 
ual’s  daily  plans,  attend  to  and  reason  about  the  execution  of 
those  plans,  and  make  flexible  and  adaptive  decisions  about 
when  it  is  most  appropriate  to  issue  reminders.  The  pa¬ 
per  concentrates  on  one  of  Autominder’s  three  main  compo¬ 
nents,  the  Plan  Manager;  other  papers  in  this  volume  describe 
its  other  components  (Colbry,  Peintner,  &  Pollack  2002; 
McCarthy  &  Pollack  2002). 

Introduction 

The  world’s  population  is  aging.  The  trend  in  the  United 
States  is  typical  of  many  industrialized  countries.  Eigures  1 
-  3  present  populations  pyramids  based  on  U.S.  census  data 
from  2000,  and  projections  for  2025  and  2050,  respectively 
(Census  2000).  Within  a  population  pyramid,  each  horizon¬ 
tal  bar  represents  the  percentage  of  U.S.  residents  in  a  five- 
year  age  cohort:  the  bottom  bar  represents  people  aged  0  to 
5;  the  bar  above  that  represents  people  aged  5  to  10;  and  so 
on,  up  to  the  topmost  bar,  which  represents  people  over  the 
age  of  100.  The  population  in  each  age  cohort  is  further  di¬ 
vided  into  males,  to  the  left  of  the  midline,  and  females,  to 
the  right.  Historically,  the  shape  of  such  graphs  is  pyramidal, 
as  there  are  more  young  people  than  older  people. 

As  can  be  seen,  in  2000,  there  is  a  significant  bulge  in 
the  25-40  year  old  cohorts,  representing  the  post-war  baby 
boom,  but  the  basic  shape  remains  pyramidal,  with  many 
more  people  under  the  age  of  60  than  people  over  60.  But 
by  2025,  the  pyramid  has  fiattened  out,  with  an  increasing 
proportion  of  people  over  60,  and  the  trend  that  continues  in 
the  2050  projection. 


Copyright  (c)  2002,  American  Association  for  Artificial  Intelli¬ 
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<NP-P2)  ProlMtod  RM4d«frtPoputa«on  of  lh*Untl«d  SIMM  M  of  July  1,2000,  Middle 


Eigure  1 :  Population  Pyramid  for  the  United  States  in  2000 


(NP-P3)  Proitclad  RnKtont  Poputation  of  tho  Unitod  Stalos  m  of  July  1.  2025,  MtddI*  Sorios. 


Figure  2:  Population  Pyramid  for  the  United  States  in  2025 
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(NP-P4)  Pro>tc««d  RtsNtom  Population  of  Um  Unilad  Stataa  aa  of  July  1, 2060,  Middia  Sanaa. 


Figure  3:  Population  Pyramid  for  the  United  States  in  2050 


According  to  the  United  Nations  Population  Division,  ev¬ 
ery  region  of  the  world  is  undergoing  a  similar  demographic 
transition.  In  2000,  606  million  people,  or  approximately 
10%  of  the  world’s  population,  were  over  60;  by  2050,  this 
percentage  is  expected  to  double,  to  2  billion  people,  or 
21.4%  of  the  population.  Even  more  dramatic  will  be  the 
increase  the  percentage  of  people  over  80,  often  called  the 
“oldest  old”.  Today  there  are  69  million  people  in  this  cat¬ 
egory,  constituting  1.1%  of  the  world’s  population.  Projec¬ 
tions  show  that  by  2050  this  percentage  will  nearly  quadru¬ 
ple,  to  4%:  there  will  be  379  million  people  over  the  age  of 
80.  The  oldest  region  of  the  world  today  is  Europe,  with  a 
median  age  of  37.5;  this  is  projected  to  rise  to  49.5  by  2050 
(United  Nations  2001). 

The  aging  of  the  world’s  population  poses  a  challenge  and 
an  opportunity  to  those  of  us  who  design  technology.  Older 
adults  face  a  range  of  challenges:  physical,  social,  emo¬ 
tional,  and  cognitive.  It  is  important  to  remember  that  there 
is  not  simply  a  growing  absolute  number  of  older  adults,  but 
that  older  adults  will  constitute  an  increasingly  large  fraction 
of  the  population.  Thus,  while  it  might  be  desirable  to  help 
older  adults  meet  their  challenges  by  providing  them  with 
human  assistance,  the  reality  is  that  there  are  not  and  will 
not  be  enough  younger  people  to  provide  all  the  support  and 
assistance  needed.  An  important  question  then,  is  how  assis¬ 
tive  technology  can  supplement  human  caregivers  to  further 
enhance  the  lives  of  older  adults. 

Many  types  of  assistive  technology  have  been  developed. 
Devices  ranging  from  the  relatively  commonplace,  e.g.,  bet¬ 
ter  hearing  aids,  to  the  futuristic,  e.g.,  intelligent  wheelchairs 
(Yanco  1998),  can  help  older  individuals  meet  physical  chal¬ 
lenges.  Older  adults  can  be  supported  socially  and  emotion¬ 
ally  through  technology  that  helps  alleviate  the  isolation  that 
is  often  a  problem  for  them.  For  example,  elder-friendly 
email  systems  (Burd  ND)and  projects  such  as  the  the  Dig¬ 
ital  Family  Portrait  (Mynatt  et  al  2001),  or  the  Dude’s 
Magic  box  (Rowan  &  Mynatt  ND)  facilitate  increased  inter¬ 
action  between  an  older  person  and  his  or  her  family  mem¬ 
bers  and  friends.  This  paper  focuses  on  technology  that  can 


help  older  adults  meet  cognitive  challenges  they  may  face. 
Specifically,  it  describes  the  use  of  automated  planning  tech¬ 
nology  to  develop  cognitive  orthotics. 

The  next  section  provides  a  brief  discussion  of  one  type 
of  cognitive  decline  that  may  occur  with  aging-a  decay  in 
prospective  memory-and  discusses  the  limitations  of  many 
existing  cognitive  orthotic  systems.  Following  that,  the  pa¬ 
per  introduces  Autominder,  a  cognitive  orthotic  designed 
and  built  at  the  University  of  Michigan  using  planning 
and  plan  management  techniques.  A  description  of  Au¬ 
tominder’s  architecture  is  followed  by  a  focused  discus¬ 
sion  of  one  of  its  three  main  components:  the  plan  man¬ 
ager.  Only  brief  descriptions  of  the  other  main  components 
are  given,  because  other  papers  in  this  proceedings  provide 
more  details  of  them  (Colbry,  Peintner,  &  Pollack  2002; 
McCarthy  &  Pollack  2002).  The  paper  concludes  by  dis¬ 
cussing  other  recent  work  on  developing  intelligent  cogni¬ 
tive  orthotics,  and  then  summarizing  the  current  state  of  Au¬ 
tominder  and  our  plans  for  continued  work. 

Cognitive  Orthotics 

Cogntive  functioning  frequently  changes  with  age:  just  as 
the  body  ages,  so  does  the  mind  (Stern  &  Carstensen  2001). 
Cognitive  changes  may  be  due  to  normal  aging,  or  may 
be  the  result  of  diseases  that  occur  with  greater  frequency 
in  older  people.  One  of  the  most  common  causes  of  se¬ 
vere  cognitive  impairment,  Alzheimer’s  Disease  (A.D.),  is 
strongly  correlated  with  age:  approximately  10%  of  people 
age  65  and  older  suffer  from  A.D.,  while  20%  of  those  aged 
70-84,  and  nearly  50%  of  those  over  85  have  A.D.  (AoA 
2000).  However,  at  least  as  important  are  milder  forms  of 
cognitive  impairment  that  may  be  prior  to  and  often  distinct 
from  A.D.  The  Autominder  system  described  in  this  paper  is 
aimed  primarily  at  people  with  mild  to  moderate  cognitive 
impairment. 

One  effect  of  age-related  cognitive  decline  may  be  de¬ 
creased  prospective  memory,  leading  to  forgetfulness  about 
routine  daily  activities,  which  the  disability-research  com¬ 
munity  call  Activities  of  Daily  Living  (ADLs)  and  Instru¬ 
mental  Activities  of  Daily  Living  (I ADLs).  ADLs  include 
fundamental  tasks  such  as  eating,  drinking,  bathing,  and 
toileting,  while  lADLs  include  tasks  such  as  managing 
medicines,  managing  money,  light  housekeeping,  arranging 
transportation,  preparing  meals,  and  so  on.  Of  course,  older 
individuals  may  have  physical  difficulties  that  impede  their 
ability  to  perform  ADLs  and  I  ADLs,  but  the  technology  de¬ 
scribed  in  this  paper  is  aimed  people  whose  primary  impair¬ 
ments  are  cognitive  ones,  which  prevent  them  from  remem¬ 
bering  to  perform  these  activities.^ 

When  an  older  adult  no  longer  consistently  performs 
ADLs  and  I  ADLs,  he  or  she  may  not  be  able  to  remain  at 
home,  but  may  need  to  move  either  to  the  home  of  a  rel¬ 
ative  or  to  a  facility-based  setting  such  as  an  assisted  care 

^The  Autominder  system  is  currently  deployed  on  a  mobile 
robot,  and  in  the  future  it  may  be  possible  to  piggyback  on  the 
robot  other  functions  that  are  intended  to  help  meet  physical  chal¬ 
lenges.  For  instance,  the  robot  could  serve  as  a  delivery  system: 
fetching  medicine,  water,  eyeglasses,  mail,  and  so  on. 
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home.  It  is  generally  accepted  that  for  many  people,  post¬ 
poning  such  a  move  as  long  as  feasible  is  desirable,  because 
people  frequently  report  a  better  quality  of  life  while  they  re¬ 
main  in  their  own  homes.  Additionally,  institutionalization 
has  an  enormous  financial  cost,  which  must  be  born  by  the 
individual,  his  or  her  family,  and/or  the  government. 

A  number  of  cogntive  orthotics  have  been  proposed  over 
the  years  to  help  older  adults  adapt  to  cognitive  declines 
and  continue  satisfactory  performance  of  routine  activities. 
Not  all  cognitive  orthotics  have  been  specifically  targeted 
to  older  individuals;  some  have  instead  been  aimed  at  peo¬ 
ple  with  cognitive  impairments  resulting  from  other  causes, 
e.g.,  brain  damage  resulting  from  stroke  or  injury.  The  idea 
of  using  computer  technology  to  enhance  the  performance 
of  cognitively  disabled  people  dates  back  nearly  forty  years 
(Englebart  1963).  Early  aids  included  talking  clocks,  calen¬ 
dar  systems,  and  similar  devices  that  were  not  very  techno¬ 
logically  sophisticated;  yet  many  are  still  in  use  today.  More 
recent  efforts  at  designing  cognitive  orthotics  have  enabled 
reminders  to  be  provided  using  the  telephone  (Eriedman 
1998),  personal  digital  assistants  (Dowds  &  Robinson  1996; 
Jonsson  &  Svensk  1995)  and  pagers  (Hersh  &  Treadgold 
1994).  Research  has  also  aimed  at  improved  modeling  of 
clients’  activities,  notably  in  the  work  of  Kirsch  and  Levine 
(Kirsch  et  al.  1987),  and  in  the  PEAT  system  (Levinson 
1997).  However,  with  the  exception  of  PEAT,  which  is  dis¬ 
cussed  further  in  the  Related  Research  section  of  this  pa¬ 
per,  these  systems  generally  function  in  a  manner  similar  to 
alarm  clocks:  they  provide  alarms  for  prescribed  activities 
at  fixed  times  that  are  specified  in  advance  by  a  client  and/or 
his  or  her  caregiver.  Eor  example,  the  web  page  for  a  typical 
cognitive  orthotic,  the  “Schedule  Assistant,”  developed  and 
marketed  by  AbleLink  Technologies,  describes  its  capabili¬ 
ties  as  follows: 

To  set  up  an  appointment  or  reminder  in  Schedule  As¬ 
sistant,  caregivers  use  a  wizard  approach  to  complete 
the  process  of  recording  a  message  or  reminder,  select¬ 
ing  a  picture  prompt  to  accompany  the  message  if  de¬ 
sired,  and  setting  the  time  and  day  for  it  to  play.  The 
system  is  then  able  to  “wake  itself  up”  to  play  the  ap¬ 
pointment  message  at  the  desired  time(AbleLinkTech 
2002). 

Although  significant  attention  has  been  given  to  the  critical 
issues  of  usability  and  interface  design  in  existing  systems, 
less  emphasis  has  been  paid  to  the  process  of  carefully  mod¬ 
eling  the  client’s  plans,  attending  to  and  reasoning  about 
their  execution,  and  deciding  whether  and  when  it  is  most 
appropriate  to  issue  reminders.  Such  reasoning  is  the  focus 
of  the  Autominder  system,  described  in  the  next  section. 

Autominder 

The  Autominder  cognitive  orthotic  is  being  developed  as 
part  of  the  Initiative  on  Personal  Robotic  Assistants  for  the 
Elderly,  a  multi-university,  multi-disciplinary  research  ef¬ 
fort  conceived  in  1998.^  The  initial  focus  of  the  Initiative 

^In  addition  to  the  University  of  Michigan,  the  initiative  in¬ 
cludes  researchers  at  the  University  of  Pittsburgh  and  Carnegie 
Mellon  University. 


Eigure  4:  Pearl:  A  Mobile  Robot  Platform  for  the  Automin¬ 
der  Cognitive  Orthotic.  Photo  courtesy  of  Carnegie  Mellon 
University. 


is  to  design  an  autonomous  mobile  robot  that  can  “live” 
in  the  home  of  an  older  individual,  and  provide  him  or 
her  with  reminders  about  daily  plans.  To  date,  two  pro¬ 
totype  robots  have  been  designed  and  built  by  members 
of  the  initiative  at  Carnegie  Mellon.  The  more  recent  of 
these  robots,  named  Pearl,  is  depicted  in  Eigure  4.  Pearl 
is  built  on  a  Nomadic  Technologies  Scout  II  robot,  with  a 
custom-designed  and  manufactured  “head”,  and  includes  a 
differential  drive  system,  two  on-board  Pentium  PCs,  wire¬ 
less  Ethernet,  SICK  laser  range  finders,  sonar  sensors,  mi¬ 
crophones  for  speech  recognition,  speakers  for  speech  syn¬ 
thesis,  touch-sensitive  graphical  displays,  and  stereo  cam¬ 
era  systems  (Baltus  et  al.  2000;  Montemerlo  et  al.  2002; 
Pineau  &  Thrun  2002).  Members  of  the  Initiative  also  have 
interests  both  in  other  ways  in  which  mobile  robots  can 
assist  older  people  (e.g.,  telemedicine,  data  collection  and 
surveillance,  and  physically  guiding  people  through  their  en¬ 
vironments),  and  in  other  platforms  for  the  cognitive  orthotic 
system  (e.g.,  wearable  devices  and  aware  homes). 

One  of  the  main  software  components  of  Pearl  is  the  cog¬ 
nitive  orthotic  system  Autominder,  which  is  being  developed 
by  members  of  the  initiative  at  the  University  of  Michigan. 
Our  goal  is  to  develop  a  system  that  is  fiexible,  adaptive, 
and  responsive-and  is  thus  more  effective  than  a  glorified 
alarm  clock.  To  attain  this  goal.  Autominder  must  main¬ 
tain  an  accurate  model  of  the  client’s  daily  plan,  monitor 
its  performance,  and  plan  reminders  accordingly.  Consider, 
for  instance,  a  forgetful,  elderly  person  with  urinary  incon¬ 
tinence  who  is  supposed  to  be  reminded  to  use  the  toilet  ev¬ 
ery  three  hours,  and  whose  next  reminder  is  scheduled  for 
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Figure  5:  Autominder  Architecture 

11:00.  Suppose  that,  using  its  on-board  sensors,  our  robot 
Pearl  observes  the  person  enter  the  bathroom  at  10:40,  and 
conveys  this  information  to  Autominder,  which  concludes 
that  toileting  has  occurred.  In  this  case,  a  reminder  should 
not  be  issued  at  11:00,  as  previously  planned.  Instead,  the 
client’s  plan  must  be  adjusted,  so  that  the  next  scheduled 
toileting  occurs  approximately  three  hours  later,  i.e.,  around 
13:40.  Flexibility  is  again  essential,  because  a  strict  three- 
hour  interval  may  not  be  optimal.  For  instance  if  the  client’s 
favorite  television  program  is  aired  from  13:30  to  14:00,  it 
might  be  better  to  issue  the  reminder  at  13:25,  and  provide  a 
justfication  that  mentions  the  television  program  (e.g.,  “Mrs. 
Smith,  Why  don’t  you  use  the  toilet  now?  That  way  I  won’t 
interrupt  you  during  your  show.”) 

Autominder’s  architecture  is  depicted  in  Figure  5.  As 
shown.  Autominder  has  three  main  components:  a  Plan 
Manager  (PM),  which  stores  the  client’s  plan  of  daily  ac¬ 
tivities  in  the  Client  Plan,  and  is  responsible  for  updating  it 
and  identifying  any  potential  conflicts  in  it;  a  Client  Modeler 
(CM),  which  uses  information  about  the  client’s  observable 
activities  to  track  the  execution  of  the  plan,  storing  its  beliefs 
about  the  execution  status  in  the  Client  Model,  and  a  Per¬ 
sonal  Cognitive  Orthotic  (PCO),  which  reasons  about  any 
disparities  between  what  the  client  is  supposed  to  do  and 
what  he  or  she  is  doing,  and  makes  decisions  about  when  to 
issue  reminders. 

Plan  Management  in  Autominder 

In  Autominder,  as  in  most  automated  planning  systems,  we 
model  plans  as  4-tuples,  <  S,0,L,B  >,  where  S  are 
steps  in  the  plans,  and  0,1/,  and  B  are  temporal  order¬ 
ing  constraints,  causal  links,  and  binding  constraints  over 
those  steps. ^  For  this  application,  temporal  constraints  are 
very  important,  and  a  rich  class  of  such  constraints  much 

^In  the  current  version  of  Autominder,  we  work  with  a  propo¬ 
sitional  representation,  and  thus  omit  binding  constraints.  On  the 
other  hand,  we  have  an  extended  class  of  links  allowed:  in  addition 


be  supported;  specifically,  we  use  the  language  of  dis¬ 
junctive  temporal  problems  (DTPs)  (Oddi  &  Cesta  2000; 
Stergiou  &  Koubarakis  2000;  Tsamardinos  2001;  Tsamardi- 
nos  &  Pollack  2002)  which  allows  for  both  quantitative 
(metric)  and  qualitative  (ordering)  constraints,  as  well  as 
conjunctive  and  disjunctive  combinations  of  these.  We  have 
also  recently  developed  an  approach  to  handling  conditional 
constraints  (Tsamardinos,  Vidal,  &  Pollack  2002),  but  we 
have  not  yet  implemented  these  in  the  Autominder  PM. 

Formally,  each  ordering  constraint  has  the  form 

Ibi  <  Xi  —  Yi  <  ubi  V  ...  V  Ibn  <  Xn  —Yn<  ubn 

where  the  Xi  and  Yi  refer  to  the  start  or  end  points  of  steps 
in  the  plan,  and  the  lower  and  upper  bounds  (Ibi  and  ubi)  are 
real  numbers.  (Without  loss  of  generality,  we  will  assume 
in  this  paper  that  they  are  integers.)  Figure  6  shows  how 
such  constraints  can  be  used  to  express  the  time  at  which 
a  step  starts  or  ends,  the  duration  of  a  step,  the  amount  of 
time  between  steps,  and  so  on,  as  well  as  expressing  ranges 
and/or  disjunctions  over  such  values.  Throughout  this  pa¬ 
per,  the  start  of  a  step  A  will  be  denoted  As  and  its  end  will 
be  denoted  Ae.  Note  that  to  express  a  clock-time  constraint, 
e.g.,  TV  watching  beginning  at  18:00,  we  use  a  temporal  ref¬ 
erence  point  (TR),  a  distinguished  value  representing  some 
fixed  clock  time.  In  the  figure,  as  well  as  in  the  Autominder 
system  itself,  the  TR  corresponds  to  midnight;  the  schedule 
is  updated  each  day. 

Note  also  how  the  disjunctive  constraints  can  be  used  to 
express  the  fact  that  two  steps  cannot  overlap.  We  illus¬ 
trate  this  further  in  Figure  7,  which  shows  a  DTP  network 
representing  the  temporal  constraints  for  a  very  small  plan. 
The  nodes  in  the  network  represent  the  start  and  end  points 
of  each  step  in  the  plan,  plus  the  temporal  reference  point, 
while  the  arcs  represent  the  nondisjunctive  constraints.  The 
one  disjunctive  constraint  is  used  to  enforce  the  fact  that  the 
two  steps  in  the  plan  cannot  overlap.  It  should  be  clear  from 
this  example  that  disjunctive  constraints  also  can  be  used  to 
express  alternative  temporal  means  of  resolving  a  conflict  in 
a  plan,  i.e.,  we  can  represent  the  possibility  of  promotion  or 
demotion  in  one  constraint. 

Plan  Initialization 

The  PM  in  Autominder  is  initialized  in  advance  of  its  use 
with  a  specification  of  the  client’s  daily  plan,  which  is  con¬ 
structed  by  the  client’s  caregiver,  possibly  in  consultation 
with  the  client  him-  or  herself.  Different  daily  plans  might 
be  constructed,  e.g.,  one  for  weekdays  and  one  for  week¬ 
ends,  with  the  appropriate  plan  loaded  each  morning,  but 
here  we  will  assume  that  there  is  just  one  daily  plan. 

We  currently  have  a  rather  minimal  GUI  for  specifying 
a  daily  plan.^  It  allows  one  to  select  pre-constructed  plan 
fragments  for  routine  activities  from  a  library,  and  to  then  in¬ 
put  specific  temporal  constraints  on  the  steps  in  the  selected 
fragments.  Thus,  a  caregiver  might  begin  construction  of  a 
typical  daily  plan  by  performing  the  following  steps: 

to  traditional  causal  links,  we  also  have  implemented  inconditions 
and  (simple)  resource  constraints. 

"^The  same  GUI  can  be  used  for  modifying  the  plan  once  exe¬ 
cution  has  begun. 
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“Toileting  should  begin  between  11:00  and  11:15.” 
660  <  Toileting s  —  TR  <  675 
“Toileting  takes  between  1  and  3  minutes.” 

1  <  ToiletingE  —  Toiletings  <  3 

“Watching  the  TV  news  can  begin  at  18:00  or  23:00.” 

1800  <  WatchNewss  —  TR  <  1802V 

2300  <  WatchNewss  —  TR<  2302 

“The  news  takes  exactly  30  minutes.” 

30  <  WatchNewsE  —  WatchNewss  <  30 
“Medicine  should  be  taken  within  1  hour  of 
finishing  breakfast.” 

0  <  TakeMedss  —  EatBreakfastE  <  60 
“Toileting  and  watching  the  news  cannot  overlap.” 

0  <  WatchNewss  —  ToiletingE  <  ooV 
0  <  Toiletings  —  WatchNewsE  <  oo 

Figure  6:  Examples  of  the  use  of  DTP  Constraints 


Sample  Temporal  Plan 

•Eat  breakfast,  starting  between  7:00  and  8:00;  it  will  take  20  to  30 
minutes. 

•Bathe,  which  will  take  30  to  40  minutes. 

•Complete  breakfast  and  bath  by  8:30. 

•Breakfast  and  bathing  may  not  overlap  temporally. 

[O^MOL^^- - ^ 

VTVTn  Disjunction  constraint: 


TR  )^420,480] 


[30,40] 


0  <  Eatg  -  Bathg  V 
0  <  Bathg  -  Eatg 


Figure  7:  Temporal  Network  for  a  Sample  Plan.  Note  the 
disjunctive  constraint  that  blocks  the  steps  from  overlapping. 


•  Select  a  pre-constructed  plan  fragment  for  breakfast, 
which  includes  three  steps-going  to  the  kitchen,  making 
breakfast,  and  eating  breakfast-as  well  as  temporal  con¬ 
straints  that  order  these,  causal  links  that  capture  their  de¬ 
pendencies,  and  some  default  durations,  e.g.,  that  the  eat¬ 
ing  step  will  take  between  20  and  30  minutes. 

•  Specify  that  the  first  step  in  the  breakfast  plan  must  begin 
by  7:00,  and  that  the  last  step  must  be  done  by  8:30. 

•  Select  a  pre-constructed  plan  fragment  for  taking 
medicine,  which  we  will  suppose  has  only  one  step-take 
the  medicine-with  a  default  duration  of  1  minute. 

•  Specify  an  interstep  constraint  to  ensure  that  the  medicine 
taking  occurs  ate  least  two  hours  after  finishing  breakfast. 

As  each  pre-constructed  plan  fragment  or  constraint  is 
added,  the  PM  performs  step  merging  (Tsamardinos,  Pol¬ 
lack,  &  Horty  2000;  Yang  1997),  that  is,  it  checks  to  ensure 
the  consistency  of  the  daily  plan  being  constructed  and  re¬ 
solves  any  conflicts.  To  do  this,  it  uses  the  same  techniques 
for  consistency  checking  that  are  used  during  plan  execu¬ 
tion;  these  techniques  are  described  in  the  next  subsection. 

Although  our  current  interface  is  sufficient  for  develop¬ 
ment  and  testing  purposes,  it  seems  clear  that  further  work 
is  required  to  develop  more  user-friendly  interfaces  to  allow 
caregivers  to  specify  plans.  Little  work  has  been  done  on 
this  topic,  but  see  (Miksch  et  al  1998)  for  one  example  of 
the  kinds  of  interfaces  that  might  be  developed. 

It  is  worth  stressing  that  the  PM  is  not  a  traditional  plan- 
generation  system.  For  the  kinds  of  routine  activities  that 
we  need  to  represent  in  our  cognitive  orthotic,  there  seems 
to  be  little  need  to  perform  planning  from  scratch.  Instead, 
it  is  sufficient  and  more  efficient  to  construct  generic  plan 
fragments,  and  allow  the  PM  to  merge  these  fragments,  a 
process  that  involves  adding  new  constraints,  but  not  new 
steps  or  causal  links.  In  future  versions  of  the  system,  we 
may  extend  the  PM  to  do  full-fledged  planning  or  replanning 
when  necessary. 

Plan  Update 

The  primary  role  of  the  PM  is  to  update  the  client’s  plan 
as  the  day  progresses,  ensuring  its  continued  consistency. 
Update  occurs  in  response  to  four  types  of  events: 

1 .  The  addition  of  a  new  activity  to  the  plan.  The  daily 
plan  created  at  initialization  provides  a  starting  point  for 
daily  activities,  but  during  the  course  of  the  day,  the  client 
and/or  his  or  her  caregivers  may  want  to  make  additions 
to  the  plan:  for  instance,  to  attend  a  bridge  game  or  a 
newly  scheduled  doctor’s  appointment.  At  this  point,  plan 
merging  must  be  performed  to  ensure  that  the  overall  plan 
remains  consistent.  Suppose  that  the  client  plan  initally 
specifies  taking  medicine  sometime  between  14:00  and 
15:00,  and  that  the  client  then  adds  a  bridge  game  out¬ 
side  the  apartment,  to  begin  at  14:30.  The  PM  must  up¬ 
date  the  plan  so  that  the  medicine-taking  step  precedes 
the  client  leaving  for  the  bridge  game.  (We  assume  that 
the  medicine  must  be  taken  at  home.)  If,  in  addition,  the 
medicine-taking  must  occur  at  least  two  hours  after  each 
meal,  the  added  restriction  on  when  the  medicine  will  be 
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taken  may  also  further  restrict  the  time  at  which  lunch 
should  be  eaten. 

2.  The  modification  or  deletion  of  an  activity  in  the  plan. 
This  is  similar  to  the  previous  case:  the  bridge  game  might 
be  cancelled,  or  the  doctor’s  office  may  change  the  time 
of  the  appointment.^  The  types  of  required  changes  are 
like  those  needed  when  an  activity  is  added.  Note  that 
the  PM  will  add  or  tighten  constraints  if  needed,  but  will 
not  “roll  back”  (i.e.,  weaken)  any  constraints.  Continuing 
the  example  above,  if  the  bridge  game  were  cancelled,  the 
constraint  that  the  medicine  be  taken  between  14:00  and 
14:30  would  remain  in  the  plan.  More  sophisticated  plan 
retraction  is  an  area  of  future  research. 

3.  The  execution  of  an  activity  in  the  plan.  The  PM  inter¬ 
acts  with  another  component  of  Autominder,  the  Client 
Modeler  (CM).  The  CM  is  tasked  with  monitoring  plan 
execution.  It  receives  reports  of  the  robot’s  sensor  read¬ 
ings,  for  instance  when  the  client  moves  from  one  room 
to  another,  and  uses  that  to  infer  the  probability  that  par¬ 
ticular  steps  in  the  client  plan  have  been  executed;  it  can 
also  issue  questions  to  the  client  for  confirmation  about 
whether  a  step  has  been  executed.  When  the  CM  believes 
with  probability  exceeding  some  threshold  that  a  given 
step  has  begun  or  ended,  it  passes  this  information  on  to 
the  PM.  The  PM  can  then  update  the  client  plan  accord¬ 
ingly.  Suppose  again  that  medicine-taking  is  supposed  to 
occur  at  least  two  hours  after  the  completion  of  each  meal. 
Upon  learning  that  breakfast  has  been  completed  at  7:45, 
the  PM  can  establish  an  earliest  start  time  of  9:45  for  tak¬ 
ing  the  medicine. 

4.  The  passage  of  a  time  boundary  in  the  plan.  Just  as  the 
execution  of  a  plan  step  may  necessitate  plan  update,  so 
may  the  non-execution  of  a  plan  step.  As  a  very  simple 
example,  suppose  that  the  client  wants  to  watch  the  news 
on  television  each  day,  either  from  18:00-18:30  or  from 
23:00-23:30  p.m.  At  18:00  (or  a  few  minutes  after),  if 
the  client  has  not  begun  watching  the  news,  then  the  PM 
should  update  the  plan  to  ensure  that  the  23:00-23:30  slot 
is  reserved  for  that  purpose.  (To  keep  the  example  sim¬ 
ple,  assume  that  the  client  always  wants  to  watch  from 
the  very  beginning  of  the  show.) 

To  perform  plan  update  in  each  of  these  cases,  the  PM  for¬ 
mulates  and  solves  a  disjunctive  temporal  problem  (DTP).  A 
DTP  is  a  constraint-satisfaction  problem  <  U,  C  >  where 
the  constrained  variables  V  represent  time  points-in  this 
case,  points  corresponding  to  the  start  and  end  of  steps- 
and  the  constraints  C  are  DTP-constraints,  as  defined  earlier 
(i.e.,  disjunctions  over  differences  between  time  points).  The 
domains  for  the  constrained  variables  are  integers,  which  in 
Autominder  represent  the  distance  in  minutes  of  the  time 
points  from  the  temporal  reference  point.  For  example,  a 
time  of  480  might  be  assigned  to  the  time  point  that  rep¬ 
resents  the  beginning  of  breakfast;  this  would  correspond 

^Currently,  we  allow  arbitrary  changes  to  be  made  to  the  plan. 
In  subsequent  versions  of  the  system,  we  will  need  to  implement 
security  mechanisms  that,  for  instance,  allow  the  user  to  make 
changes  to  social  engagements  but  not  the  medicine-taking  actions. 


Update-Plan-for- Addition(exi  5  frag) 

E  =  ConYeYt-to-DTF(existing) 

N  =  Convert-to-DTP(neu; /ra^) 

C  =  Identify-confiicts(exi5tm^  U  new  frag) 
R  =  0 

For  each  member  c  of  C 

R  =  R  U  a  DTP-constraint  representing 
the  alternative  temporal  resolutions  of  c 
P=EUNUR 
P’  =  Solve-DTP(P) 

Return(Convert-to-Plan-Representation(P’)) 


Figure  8:  Algorithm  for  Update  after  a  Plan  Addition 

to  8:00  (480  minutes  after  the  temporal  reference  point  of 
midnight).  In  fact,  we  do  not  need  to  assign  exact  times  to 
most  time  points;  instead  we  find  solutions  that  correspond 
to  maximum  allowable  time  intervals. 

To  see  how  this  works,  consider  first  the  case  of  updating 
the  plan  in  response  to  a  plan  addition.  Psuedo-code  for  this 
case  is  given  in  Figure  8.  The  PM  begins  with  the  contents  of 
the  Client  Plan,  existing,  and  a  plan  fragment  representing 
the  new  activities  to  be  added  to  the  plan,  new  frag.  Both 
existing  and  new  frag  are  encoded  as  <  S,0,L,B  >  4- 
tuples,  and  so  the  first  step  is  to  convert  them  to  disjunctive 
temporal  problems,  E  and  N,  respectively.  This  is  a  triv¬ 
ial  process  that  is  linear  in  the  number  of  steps:  it  involves 
simply  extracting  all  the  temporal  constraints  and  encoding 
them  in  a  format  that  our  DTP  solving  engine  can  handle. 
Note  that  there  is  information  lost  in  the  DTP  encoding: 
specifically,  the  DTP  does  not  encode  causal  links.  Thus, 
it  is  crucial  that  a  temporal  constraint  be  explicitly  included 
for  each  causal  link.  Additionally,  it  is  necessary  to  iden¬ 
tify  all  the  threats  in  the  union  of  existing  and  new  frag, 
a  process  that  is  quadratic  in  the  total  number  of  steps.  For 
each  identified  threat,  the  PM  then  constructs  a  DTP  con¬ 
straint  that  represents  the  alternative  methods  of  resolution; 
call  the  set  of  such  threat-resolution  constraints  R.  Finally, 
a  plan  P  that  consists  of  the  union  of  E,N  and  R  is  passed 
to  a  DTP-solver,  which  checks  for  consistency,  and  returns 
P  augmented  by  a  set  of  additional  constraints  that  ensure 
consistency.  In  particular,  if  there  are  any  threats  in  the  plan, 
a  resolution  will  be  selected  for  each  one.  The  last  step  in 
the  process  is  to  convert  the  new  set  of  DTP  constraints  back 
to  a  plan  tuple. 

DTP  solving,  which  is  NP-complete,  is  the  only  com¬ 
putationally  expensive  step  in  the  process.  In  Automin¬ 
der,  we  use  the  Epilitis  DTP-solver  (Tsamardinos  2001; 
Tsamardinos  &  Pollack  2002).  Epilitis  integrates  a  number 
of  efficiency  heuristics,  and  has  been  demonstrated  to  solve 
benchmark  problems  two  orders  of  magnitude  faster  than  the 
previous  state-of-the  art  solvers.  For  our  current  Autominder 
scenarios,  which  typically  involve  about  30  actions,  Epilitis 
nearly  always  produces  solutions  in  less  than  one  second,  a 
time  that  is  well  within  the  bounds  we  require. 

Like  prior  DTP  solvers  (Oddi  &  Cesta  2000;  Stergiou 
&  Koubarakis  2000;  Armando,  Castellini,  &  Giunchiglia 
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Update-Plan-for-Modification(exi5tm^,  mods) 

plan  =  Make  the  modifications  in  mods  to  existing 
(i.e.,  remove  and/or  replace  constraints) 

M  =  Convert- to-DTP(p/ an) 

C  =  Identify-confiicts(p/an) 

R  =  0 

For  each  member  c  of  C 

R  =  R  U  a  DTP-constraint  representing 
the  alternative  temporal  resolutions  of  c 
P  =  MUR 
P’  =  Solve-DTP(P) 

Retum(Convert-to-Plan-Representation(P’)) 


Figure  9:  Algorithm  for  Update  after  a  Plan  Modification 


1999),  Epilitis  does  not  attempt  to  solve  the  DTP  directly  by 
searching  for  an  assignment  of  integers  to  the  time  points. 
Instead,  it  solves  a  meta-CSP  problem:  it  attempts  to  find 
one  disjunct  from  each  disjunctive  constraint  such  that  the 
set  of  all  selected  disjuncts  forms  a  consistent  Simple  Tem¬ 
poral  Problem  (STP)  (Dechter,  Meiri,  &  Pearl  1991).  An 
STP  is  like  a  DTP,  except  that  the  constraints  must  be  atomic 
inequalities;  no  disjunctions  are  allowed.  The  details  are  be¬ 
yond  the  scope  of  the  current  paper  (but  see  (Tsamardinos 
2001;  Tsamardinos  &  Pollack  2002)).  The  important  point 
here  is  that  by  using  this  approach,  Epilitis  can  return  an 
entire  STP,  which  provides  interval  rather  than  exact  con¬ 
straints  on  the  time  points  in  the  plan.  Consider  again  our 
example  of  the  plan  that  involves  taking  medicine  between 
14:00  and  15:00,  which  is  amended  with  a  plan  to  leave  for 
a  bridge  game  at  14:30.  Epilitis  will  return  a  DTP  that  con¬ 
strains  the  the  medicine  to  be  taken  sometime  between  14:00 
and  14:30;  it  does  not  have  to  assign  a  specific  time  (e.g., 
14:10)  to  that  action. 

The  other  three  cases  of  plan  update  are  similar.  In  re¬ 
sponse  to  a  plan  modification,  the  PM  again  begins  with  the 
current  contents  of  the  Client  Plan,  existing,  but  this  time, 
instead  of  a  second  plan  to  merge  in,  it  has  a  set  of  con¬ 
straints  from  existing  that  are  to  be  removed  or  changed. 
Thus,  it  makes  the  specified  modifications  to  existing  and 
then  converts  it  to  a  DTP,  identifies  conflicts,  and  performs 
DTP  solving  as  before.  The  pseudo-code  for  this  is  shown 
in  Figure  9. 

The  psuedo-code  for  the  other  two  cases  of  plan  update 
is  not  shown,  as  they  are  similar  to  the  previous  ones.  In 
the  third  case  of  update,  a  step  S  has  begun  or  finished 
execution.  In  response,  the  PM  shrinks  the  temporal  con- 
straint(s)  associated  with  the  start  end,  and/or  duration  of  S 
to  a  unit  interval.  For  instance,  if  we  know  that  breakfast 
began  at  time  480,  then  the  constraint  associated  with  it  be¬ 
comes  480  <  EatBreakfasts  —  TR  <  480.  As  long  as 
execution  has  occurred  within  the  legal  bounds,  there  is  no 
need  to  identify  conflicts;  instead,  the  resulting  plan  with  the 
reduced  constraints  is  passed  directly  to  the  DTP  solver  so 
that  the  new  tighter  constraints  can  be  propagated. 

In  the  fourth  case,  a  time  boundary  has  passed  without 
a  step  having  begun  or  ended.  At  this  point,  the  PM  must 


remove  the  now  invalidated  disjunct  from  a  constraint,  and 
then  attempt  to  solve  the  DTP  anew.  In  our  TV  news  exam¬ 
ple,  the  plan  would  include  a  constraint 
1800  <  WatehNewss  —  TR<  1802V 
2300  <  WatehNewss  —  TR<  2302 
i.e.,  that  watching  the  news  must  start  either  right  about 
18:00,  or  else  about  23:00.  If  this  step  has  not  begun  by 
shortly  after  18:00,  the  first  disjunct  is  no  longer  viable. 
Thus,  the  PM  must  remove  it  from  the  representation  of  the 
plan,  and  attempt  to  resolve  the  DTP,  using  the  remaining 
disjunct.  In  the  current  example,  there  is  an  alternative  dis¬ 
junction  to  try.  Sometimes,  though,  when  an  invalidated  dis¬ 
junct  is  removed,  there  may  not  remain  any  alternatives;  in 
that  case  an  execution  failure  has  occurred.  As  with  other 
cases  of  execution  failure,  e.g.,  missed  deadlines.  Automin¬ 
der  would  record  this  fact,  making  it  available  to  the  care¬ 
giver  if  appropriate. 

The  discussion  of  passed  time  boundaries  brings  to  light 
one  point  that  was  passed  over  earlier.  In  general,  there  may 
be  multiple  solutions  to  a  DTP,  i.e.,  multiple  consistent  STPs 
that  can  be  extracted  from  the  DTP.  In  the  current  version  of 
Autominder,  the  PM  arbitrarily  selects  one  of  these  (the  first 
one  it  finds).  If  subsequent  execution  is  not  consistent  with 
the  STP  selected,  then  the  DTP  will  attempt  to  find  an  al¬ 
ternative  consistent  solution.  A  more  principled  approach 
would  select  solutions  in  an  order  that  provides  the  greatest 
execution  flexibility.  For  example,  the  solution  that  involves 
watching  the  18:00  news  leaves  open  the  possibility  of  in¬ 
stead  watching  the  news  at  23:00.  If  the  first  solution  found 
instead  involved  watching  the  later  news  show,  then  after 
an  execution  failure  there  would  be  no  way  to  recover,  as  it 
would  be  too  late  to  watch  the  18:00  news.  Unfortunately 
selecting  DTP  solutions  to  maximize  flexibility  is  a  difficult 
problem  (Tsamardinos,  Pollack,  &  Ganchev  2001). 

Other  Autominder  Components 

In  addition  to  the  PM,  Autominder  has  two  other  princi¬ 
pal  components.  The  Client  Modeler  (CM)  was  mentioned 
above  in  the  discussion  on  updating  the  plan  in  response  to 
plan  execution.  As  noted  there,  the  job  of  the  CM  is  to  mon¬ 
itor  the  execution  of  the  plan,  attempting  to  infer  its  status 
from  information  obtained  from  the  robot  sensors  and  re¬ 
questing  confirmation  from  the  client  when  appropriate.  To 
build  the  CM,  we  have  been  adapting  Bayesian  inference 
mechanisms  to  handle  the  temporal  demands  of  this  appli¬ 
cation;  details  can  be  found  in  (Colbry,  Peintner,  &  Pollack 
2002). 

The  remaining  component  of  Autominder  is  the  Personal 
Cognitive  Orthotic  (PCO),  which  is  responsible  for  mak¬ 
ing  the  decision  about  what  reminders  to  issue  and  when. 
To  do  this,  the  PCO  reasons  about  the  client  plan  and  the 
client  model,  identifying  any  evolving  discrepancies  be¬ 
tween  them.  It  turns  out  to  be  relatively  easy  to  generate  a 
legal  reminder  plan-such  a  plan  simply  includes  a  reminder 
for  every  planned  activity  at  the  earliest  possible  time  of  its 
execution.  However,  a  reminder  plan  constructed  this  way  is 
likely  to  be  a  rather  poor  one  when  judged  by  the  criteria  we 
use  in  Autominder,  namely: 
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1 .  ensuring  that  the  client  is  aware  of  activities  he  or  she  is 
expected  to  perform, 

2.  increasing  the  likelihood  that  the  client  will  perform  at 
least  the  essential  activities  (such  as  taking  medicine), 

3.  avoiding  annoying  the  client,  and 

4.  avoiding  making  the  client  overly  reliant  on  the  system. 

While  the  simple  approach  would  lead  to  a  reminder  plan 
that  satisfies  the  first  criteria,  it  is  unlikely  to  satisfy  the  third 
or  fourth,  and  this  in  turn  may  have  a  negative  impact  on  the 
second  criteria.  Consequently,  we  employ  the  local  search 
techniques  of  the  Planning  by  Rewriting  algorithm  (Ambite 
&  Knoblock  2001)  to  iteratively  search  for  an  improved  re¬ 
minder  plan;  for  details  of  our  approach,  see  (McCarthy  & 
Pollack  2002) 

Related  Research 

Several  existing  cognitive  orthotics  systems  were  mentioned 
earlier  in  this  paper.  The  most  notable  of  these  from  a  plan- 
based  perspective  is  PEAT  (Levinson  1997).  This  was  the 
first,  and  to  the  best  of  our  knowledge,  the  only  marketed 
cognitive  orthotic  system  that  relies  on  automated  planning 
technology.  PEAT,  which  is  marketed  primarily  to  patients 
with  traumatic  brain  injury,  is  deployed  on  a  handheld  de¬ 
vice,  and  provides  visible  and  audible  clues  about  plan  exe¬ 
cution.  Like  Autominder,  PEAT  maintains  a  detailed  model 
of  the  client’s  plan  and  tracks  its  execution,  propagating  tem¬ 
poral  constraints  when  the  client  inputs  information  specify¬ 
ing  that  an  action  has  been  performed.  Also,  upon  the  addi¬ 
tion  of  a  new  action,  PEAT  simulates  the  plan  to  uncover  any 
conflicts,  using  the  PROPEL  planning  and  execution  system 
(Levinson  1995)  for  this  purpose.  However,  PEAT  uses  a 
less  expressive  planning  language  than  Autominder;  it  does 
not  attempt  to  infer  the  plan  execution  status;  and  it  does  not 
perform  principled  reasoning  about  what  reminders  to  issue 
when,  instead  automatically  providing  a  reminder  for  each 
planned  activity. 

Within  the  past  year  or  two,  several  new  projects  aimed 
at  designing  intelligent  cognitive  orthotics  have  begun  to 
emerge.  The  MAPS  project  at  the  University  of  Colorado 
is  focusing  on  the  HCI  issues  involved  in  building  a  hand¬ 
held  cognitive  orthotic  (Carmien  2002).  The  Independent 
Lifestyle  Assistant  Project  (ILSA)  at  Honeywell  is  another 
recent  related  effort,  which  has  some  aims  that  overlap  with 
our  own  (Miller  &  Riley  2001).  Yet  another,  even  newer 
project  is  the  Assisted  Cognition  Project  at  the  University 
of  Washington  (Kautz  et  al.  2002).  While  Autominder  is 
being  targeted  mainly  at  people  with  milder  forms  of  cogni¬ 
tive  impairment,  the  Washington  project  aims  at  developing 
a  cognitive  orthotic  system-an  adaptive  prompter-for  peo¬ 
ple  with  Alzheimer’s  disease.  The  system  will  use  ubiqui¬ 
tous  sensors  to  monitor  the  performance  of  routine  tasks,  and 
provide  prompts  when  a  client  gets  “stuck”.  Eor  instance,  a 
sensor  in  the  bathroom  might  notice  that  a  person  with  A.D. 
has  picked  up  a  toothbrush  but  then  stopped;  in  response, 
the  adaptive  prompter  would  provide  guidance  to  the  person 
about  putting  toothpaste  on  the  brush  and  using  it  to  brush 
his  or  her  teeth.  As  can  be  seen,  the  adaptive  prompter  is 


targeted  at  people  with  more  severe  cognitive  decline  than 
what  we  imagine  for  a  typical  Autominder  client. 

Conclusions 

The  Autominder  system  as  described  in  this  paper  has  been 
fully  implemented  in  Java  and  Lisp  on  Wintel  platforms;  we 
are  also  working  on  a  Web-based  interface  for  plan  initial¬ 
ization  and  update.  The  most  recent  version  system  has  been 
tested  in  the  laboratory;  an  earlier  version  was  integrated 
with  the  robot  software  and  included  in  a  preliminary  field 
test  conducted  at  the  Longwood  Retirement  Community  in 
Oakmont,  PA  in  June,  2001.  The  goals  of  that  test  were, 
first,  to  ensure  that  the  robot  control  software  and  the  cog¬ 
nitive  orthotic  would  work  together,  and  second,  to  get  an 
initial  sense  of  the  acceptability  of  such  a  system  to  older 
individuals.  On  both  accounts,  the  test  was  successful.  Ad¬ 
mittedly,  the  older  adults  who  enrolled  in  the  studies  were 
volunteers,  and  people  likely  to  be  intimidated  or  put  off  by 
this  type  of  technology  would  not  have  volunteered.  How¬ 
ever,  the  people  who  did  participate  were  uniformly  excited 
about  the  system,  as  were  the  staff  at  Longwood,  who  made 
a  number  of  suggestions  to  us  about  how  this  type  of  tech¬ 
nology  could  also  be  used  to  assist  them  in  their  caregiving 
tasks.  We  intend  to  conduct  interviews  later  this  year  with 
caregivers  and  residents  at  Longwood  in  order  to  develop 
more  detailed  models  of  the  daily  plans  of  several  residents, 
and  then  to  field  test  a  version  of  Autominder  that  encodes 
those  plans.  These  field  tests  will  be  more  directly  focused 
on  the  performance  of  the  cognitive  orthotics  software. 

We  have  a  number  of  plans  for  the  continued  develop¬ 
ment  of  Autominder,  some  of  which  were  already  mentioned 
in  this  paper.  We  have  planned  extensions  to  the  individ¬ 
ual  reasoning  modules,  for  example,  adding  the  ability  to 
handle  conditional  constraints  to  the  PM;  supplementing  the 
PM  with  full-fledged  planning  capabilities  to  support  replan¬ 
ning;  enabling  the  CM  to  learn  the  patterns  of  client  activity 
over  time,  in  order  to  better  interpret  observed  behavior;  and 
developing  techniques  for  providing  better  justifications  for 
reminders  issued  by  the  PCO.  We  are  also  interested  in  the 
deployment  of  the  system  on  alternative  hardware  platforms. 
Although  there  are  many  advantages  to  using  a  robot,  in¬ 
cluding  the  ability  to  piggyback  on  other  capabilities,  there 
are  clearly  also  reasons  to  explore  handheld  and/or  wearable 
devices  and  ubiqitous  sensors  to  support  cognitive  orthotics. 
Linally,  after  our  experiences  with  the  staff  at  Longwood, 
we  are  interested  in  exploring  the  use  of  systems  like  ours 
within  the  facility-based  setting.  In  that  context,  the  system 
would  coordinate  the  daily  plans  not  only  of  a  single  person, 
but  of  multiple  people,  including  both  the  residents  and  the 
staff  that  takes  care  of  them. 
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Abstract 

Temporal  uncertainty  is  a  feature  of  many  real-world  plan¬ 
ning  problems.  One  of  the  most  successful  formalisms  for 
dealing  with  temporal  uncertainty  is  the  Simple  Temporal 
Problem  with  uncertainty  (STP-u).  A  very  attractive  fea¬ 
ture  of  STP-u's  is  that  one  can  determine  in  polynomial 
time  whether  a  given  STP-u  is  dynamically  controllable,  i.e., 
whether  there  is  a  guaranteed  means  of  execution  such  that  all 
the  constraints  are  respected,  regardless  of  the  exact  timing 
of  the  uncertain  events.  Unfortunately,  if  the  STP-u  is  not  dy¬ 
namically  controllable,  limitations  of  the  formalism  prevent 
further  reasoning  about  the  probability  of  legal  execution.  In 
this  paper,  we  present  an  alternative  formalism,  called  Prob¬ 
abilistic  Simple  Temporal  Problems  (PSTPs),  which  general¬ 
izes  STP-u  to  allow  for  such  reasoning.  We  show  that  while 
it  is  difficult  to  compute  the  exact  probability  of  legal  exe¬ 
cution,  there  are  methods  for  bounding  the  probability  both 
from  above  and  below,  and  we  sketch  alternative  candidate  al¬ 
gorithms  for  this  purpose.  Computing  the  probability  of  legal 
execution  allows  a  temporal  planner  to  decide,  when  uncer¬ 
tainty  is  present,  whether  to  accept  or  reject  candidate  plans. 

In  addition,  lower  bound  computation  has  an  important  side- 
effect:  it  provides  guidance  as  to  how  to  execute  an  STP-u 
even  when  it  is  not  dynamically  controllable. 

Introduction 

Many  real-world  planning  problems  involve  temporal  con¬ 
straints,  and  a  number  of  planning  formalisms  and  algo¬ 
rithms  have  been  developed  to  deal  with  them.  One  of  the 
most  well-known  is  the  Simple  Temporal  Problem  (STP) 
formalism  (Dechter,  Meiri,  &  Pearl  1991),  which  allows  the 
representation  of  temporal  constraints  of  the  form  X  —  Y  < 
d,  where  X  and  Y  are  the  times  of  occurrence  of  two  instan¬ 
taneous  events  in  the  plan  and  d  is  a  real  number  (or  infinity). 
For  example,  if  X  and  Y  denote  the  start  and  end  points  of 
a  single  action,  then  the  constraint  specifies  that  the  action 
takes  no  more  than  d  time  units. 

The  STP  formalism,  along  with  generalizations  of  it,  such 
as  the  Disjunctive  Temporal  Problem  (DTP)  (Stergiou  & 
Koubarakis  2000;  Tsamardinos  2001)  have  been  very  fruit¬ 
ful,  both  for  theoretical  investigations  of  temporal  planning 
(Smith,  Frank,  &  Jonsson  2000)  and  for  practical  deploy¬ 
ment,  notably  in  NASA’s  Remote  Agent  (Muscettola  et  al 
1998).  However,  these  formalisms  do  not  allow  any  explicit 
representation  of  uncertainty.  Yet  in  most  interesting,  real- 


world  domains,  there  are  many  types  of  uncertainty.  One 
type  of  uncertainty  is  associated  with  conditional  execution 
of  actions  that  depend  on  observations  and  the  status  of  the 
world  during  execution.  The  Conditional  Temporal  Problem 
(Tsamardinos,  Vidal,  &  Pollack  2003)  is  an  extension  of  the 
STP  that  is  able  to  encode  and  reason  with  quantitative  tem¬ 
poral  constraints  and  conditional  branches. 

Another  type  of  uncertainty,  that  this  paper  addresses, 
is  temporal  uncertainty,  i.e.,  uncertainty  about  the  time  at 
which  particular  events  will  occur.  Such  events  are  said  to 
uncontrollable,  to  distinguish  them  from  the  events  that  are 
under  the  control  of  the  agent  executing  the  plan.  Often, 
plans  must  include  temporal  constraints  that  involve  uncon¬ 
trollable  events:  for  instance,  it  may  be  necessary  to  respond 
to  an  alarm  within  two  minutes  of  its  going  off.  The  time 
of  the  alarm  is  not  within  the  control  of  the  execution  agent, 
but  the  time  of  the  responsive  event  is. 

In  order  to  model  uncontrollable  events,  an  extended 
fonualism,  called  Simple  Temporal  Problems  with  Un¬ 
certainty  (STP-u)  was  developed  (Vidal  &  Ghallab  1996; 
Vidal  &  Fargier  1997;  Morris,  Muscettola,  &  Vidal  2001), 
With  STP-u’s,  one  can  model  plans  that  contain  constraints 
involving  uncontrollable  events.  Notice  that  with  such  plans, 
decisions  about  when  to  perform  actions  must  often  be  de¬ 
ferred  until  execution  time.  For  instance,  one  cannot  decide 
in  advance  when  to  respond  to  an  alarm:  all  one  can  do  is 
wait  until  the  alarm  goes  off  and  then  respond  accordingly, 
A  very  attractive  feature  of  STP-u’s  is  that  one  can  deter¬ 
mine  in  polynomial  time  whether  a  given  STP-u  is  dynami¬ 
cally  controllable,  i,e,,  whether  there  is  a  guaranteed  means 
of  execution  such  that  all  the  constraints  are  respected,  re¬ 
gardless  of  the  exact  timing  of  the  uncertain  eventsk  Not  all 
STP-u’s  are  dynamically  controllable.  As  a  simple  exam¬ 
ple,  consider  an  STP-u  that  includes  a  constraint  requiring 
an  agent  to  respond  to  an  uncontrollable  alarm  exactly  two 
minutes  before  it  goes  off.  If  the  agent  doesn’t  control  the 
alarm-does  not  know  when  it  will  go  off  and  does  not  have 
any  means  of  making  it  go  off-then  clearly  the  agent  cannot 
act  in  a  way  to  satisfy  this  constraint, 

A  plan  generation  system  can  approach  the  task  of  plan- 


^  Here,  and  in  the  rest  of  the  paper  we  assume  the  only  source 
of  uncertainty  in  the  plan  is  the  temporal  uncertainty  of  the  uncon¬ 
trollable  events. 
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ning  under  temporal  uncertainty  by  generating  a  plan  for¬ 
mulated  as  an  STP-u  and  then  checking  to  see  whether  it  is 
dynamically  controllable.  If  it  is,  the  planner  can  declare 
success.  Unfortunately,  if  it  is  not,  limitations  of  the  STP-u 
formalism  preclude  further  reasoning  both  about  the  prob¬ 
ability  of  legal  execution,  and  about  strategies  to  maximize 
that  probability.  Consequently,  the  planning  system  does  not 
know  whether  to  adopt  the  plan  or  to  search  for  an  alternate, 
and,  if  the  former-if  it  does  adopt  the  plan-it  is  unable  to 
formulate  a  effective  means  of  executing  it. 

In  this  paper,  we  present  a  new  formalism,  called  Prob¬ 
abilistic  Simple  Temporal  Problems  (PSTPs),  which  gener¬ 
alizes  STP-u ’s  in  a  way  that  supports  reasoning  about  the 
probability  that  a  plan  with  temporal  uncertainty  will  be 
legally  executed.  Although  is  is  difficult  to  compute  an  exact 
probability,  we  show  that  there  are  methods  for  bounding  the 
probability  both  from  above  and  below.  An  upper  bound  can 
be  used  to  reject  a  current  candidate  plan  which  falls  below 
a  given  threshold,  while  a  lower  bound  can  be  used  to  accept 
a  candidate  plan. 

The  remainder  of  our  paper  is  organized  as  follows.  In 
the  Background  section  we  review  the  background  material 
on  STPs,  STP-u ’s,  and  dynamic  controllability,  and  in  the 
next  section  we  introduce  the  Probabilistic  Simple  Tempo¬ 
ral  Problems  (PSTP)  formalism.  In  the  following  two  sec¬ 
tions  we  describe  the  technique  for  computing  the  upper 
bound  and  lower  bound  on  the  probability  of  correctly  ex¬ 
ecuting  a  PSTP,  respectively.  In  particular,  we  explain  how 
the  problem  of  computing  the  lower  bound  can  be  addressed 
by  first  converting  a  PSTP  to  an  STP-u  and  then  tighten¬ 
ing  the  bounds  on  that  STP-u  until  it  becomes  dynamically 
controllable.  The  following  three  sections  then  sketch  some 
approaches  to  this  lower  bound  computation. 

Finally,  the  Discussion  section  summarizes  the  main 
ideas,  open  questions,  and  future  directions.  We  note  there 
in  particular  that  the  lower  bound  computation  has  an  impor¬ 
tant  side-effect:  it  results  in  the  specification  of  an  execution 
strategy  that  maximizes  the  probability  of  satisfying  all  the 
execution  constraints.  From  another  perspective  this  means 
that  it  provides  guidance  as  to  how  to  execute  an  STP-u  even 
when  it  is  not  dynamically  controllable. 

Background 

The  formalism  used  to  represent  temporal  information  and 
uncertainty  is  based  on  the  Simple  Temporal  Problem  (STP) 
defined  below: 

Definition  1.  Simple  Temporal  Problem,  STP  Solution, 
Consistent  STP,  A  Simple  Temporal  Problem  (STP)  is  a 
pair  <V,E  where 

•  U  is  a  set  of  variables  (also  called  nodes,  events,  or  time- 
points)  taking  real  values  representing  the  time  of  occur¬ 
rence  of  instanteneous  events. 

•  FJ  a  set  of  temporal  constraints  on  the  variables  of  the 
form  A  -  y  <  6,  A,  y  G  U,  and  6  G  5?  U  {-oo,  cx)}. 

A  solution  to  an  STP  is  an  assignment  to  the  variables  that 
satisfies  the  constraints.  An  STP  is  consistent  if  there  exists 
at  least  one  solution. 


STPs  have  been  used  to  represent  temporal  plans  by  using 
a  variable  for  each  action’s  start  and  end  point.  For  example, 
if  start{A)  and  end{A)  are  the  events  of  starting  and  ending 
action  A,  then  the  constraints  5  <  end{A)  —  start{A)  <  10 
specify  the  duration  of  the  action  to  be  between  5  and  1 0 
time  units. 

STP  constraints  are  binary.  In  order  to  represent 
unary  constraints  on  absolute  execution  time,  e.g.,  100  < 
start  (A)  <  200,  a  special  variable  called  time  reference 
point  TRis  defined  and  is  assigned  time  zero;  then,  the  con¬ 
straint  above  can  be  written  as  100  <  start{A)—TR  <  200. 

By  using  an  all-pairs  shortest  path  algorithm  one  can  dis¬ 
cover  the  distance  from  y  to  A  denoted  as  dyx  and  defined 
as  follows: 

Definition  2.  The  distance  from  variable  Y  to  variable  A 
denoted  as  dyx  is  the  smallest  number  for  which  the  equa¬ 
tion  A  -  y  <  dyx  holds  in  all  STP  solutions, 

STPs  can  be  executed  with  minimal  on-line  con¬ 
straint  propagation  as  discussed  in  (Muscettola,  Morris, 
&  Tsamardinos  1998;  Tsamardinos,  Morris,  &  Muscettola 
1988).  An  STP  does  not  represent  uncertainty  infonnation 
about  the  occurence  of  events.  All  variables  are  assumed 
to  be  under  the  direct  control  of  the  agent  executing  the 
represented  plan  and  so,  if  there  exists  a  solution,  then  the 
STP  is  executable  in  a  way  that  satisfies  its  constraints.  To 
address  this  representational  limitation,  the  Simple  Tempo¬ 
ral  Problem  with  Uncertainty  (STP-u)  formalism  was  de¬ 
veloped  (Vidal  &  Ghallab  1996;  Vidal  &  Fargier  1997; 
Moms,  Muscettola,  &  Vidal  2001). 

An  STP-u  makes  a  distinction  between  controllable  and 
uncontrollable  variables.  Controllable  variables  are  the  ones 
whose  timing  of  execution  is  under  the  direct  control  of  the 
agent.  Uncontrollable  variables  are  the  ones  whose  timing 
of  execution  depends  on  Nature  (i.e.,  exogenous  factors). 
The  only  information  represented  regarding  the  exact  timing 
of  an  uncontrollable  A  is  that  it  will  occur  sometime  within 
the  interval  [/,  u\  after  a  controllable  y,  called  the  parent  of 
A.  Thus,  the  STP-u  specifies  that  Nature  will  respect  the 
constraint  I  <  X  —  Y  <  u.  These  constraints  involving 
Nature  are  called  contingent  links  and  are  distinct  from  the 
constraints  the  plan  has  to  respect  to  be  legally  executed, 
called  requirement  links. 

Definition  3,  Simple  Temporal  Problem  with  Uncer¬ 
tainty.  A  Simple  Temporal  Problem  with  Uncertainty  STP- 
u  is  a  tuple  <Vc,E,Vu,C  >,  where 

•  Vc  and  Vu  are  the  set  of  controllable  and  uncontrollable 
variables,  respectively,  taking  real  values, 

•  is  a  set  of  constraints  (requirement  links)  of  the  form 
A  -  y  <  &,  A,  y  G  Uc  U  Ut/,  and  &  G  U  {-oo,  oo}. 

•  C  is  a  set  of  contingent  links  of  the  form  /  <  A  —  y  <u, 
Y  gVc.X  GVu.Sindl.ue^. 

Figure  4  shows  an  STP-u  with  two  controllable  variables 
A,  B  and  an  uncontrollable  variable  C.  The  edge  A  ^  B  in 
the  figure,  annotated  with  the  interval  [p,  q]  graphically  ex¬ 
presses  the  constraints  p  <  B  — A  <  g,  i.e.,  A  — ^  <  — pand 
B  —  A  <  q.  Similar  constraints  hold  for  the  other  edges.  The 
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edge  A  ^  C  is  a  contingent  link,  i.e.,  a  constraint  Nature  is 
expected  to  observe. 

Contingent  constraints  are  always  between  a  controllable 
variable  Y  and  an  uncontrollable  variable  X.  A  contingent 
link  I  <  end{A)—start{A)  <  u  may  be  used  for  example  to 
specify  that  the  expected  duration  of  an  action  A  is  between 
I  and  u  time  units;  however,  this  duration  is  not  something 
that  is  detemnined  by  the  agent. 

An  STP-u,  like  an  STP,  should  be  executed  in  such  a  way 
that  all  its  constraints  are  satisfied.  However,  as  we  illus¬ 
trated  in  the  introduction  with  the  alarm  example,  the  ex¬ 
istence  of  uncontrollable  events  means  that  decisions  about 
the  timing  of  controllable  events  may  need  to  be  deferred  to 
execution  time. 

Definition  4.  Legal  Execution,  Execution  Strategy.  A  le¬ 
gal  execution  of  a  STP-u  <  Vc-;E^Vu-;C  >  is  a  schedule 
(time  assignment)  of  occurrences  of  the  events  (variables) 
in  Vc  U  Ct/  in  a  way  that  satisfies  all  the  constraints  in  E. 
An  execution  strategy  is  an  algorithm  that  decides  when 
to  execute  the  next  controllable  action  given  the  execution 
constraints  and  the  observed  history  of  the  uncontrollable 
events. 

Definition  5.  Dynamic  Controllability.  (Informal)  An 
STP-u  <  Vc.E.Vu.C  >  is  dynamically  controllable  if 
there  exists  an  execution  strategy  that  results  in  a  legal  exe¬ 
cution  regardless  of  when  the  uncontrollable  variables  in  Vu 
occur  (provided  they  occur  within  the  bounds  specified  by 
the  contingent  constraints  in  C). 

For  a  formal  definition  of  dynamic  controllability,  see 
(Morris,  Muscettola,  &  Vidal  2001),  which  also  provides 
a  polynomial-time  algorithm  for  checking  whether  a  given 
STP-u  is  dynamically  controllable. 

How  does  a  domain  expert  model  temporal  uncertainty 
when  specifying  the  constraints  for  a  planner?  For  any  tem¬ 
porally  uncertain  event  A,  the  expert  must  specify  some 
bounds  on  the  time  of  occurrence  of  A.  In  the  STP-u  for¬ 
malism,  this  corresponds  to  setting  the  values  of  I  and  u  in 
a  contingent  link  I  <  end{A)  —  start{A)  <  u.  The  looser 
these  bounds  are  set  to  be,  the  more  likely  they  are  to  be  ob¬ 
served  by  Nature,  and  hence,  the  more  accurate  the  model 
is;  in  the  extreme  case,  if  they  are  set  to  positive  and  neg¬ 
ative  infinity,  then  the  expert  is  certain  that  the  event  will 
occur  within  the  specified  bounds.  On  the  other  hand,  as 
the  bounds  get  looser,  the  likelihood  decreases  that  the  STP- 
u  is  dynamically  controllable.  And  when  the  STP-u  is  not 
dynamically  controllable,  the  formalism  provides  no  guid¬ 
ance  about  when  to  perform  the  controllable  events  so  as  to 
increase  the  probability  of  observing  the  constraints.  Cur¬ 
rently,  there  are  no  principled  procedures  for  deciding  the 
bounds  of  the  uncontrollables  in  a  way  that  maximizes  the 
probability  that  Nature  will  respect  them  and  that  the  result¬ 
ing  STP-u  will  be  dynamically  controllable. 

In  fact,  because  STP-u's  do  not  explcitly  represent  prob¬ 
ability  distributions  of  uncontrollable  events,  they  lack  the 
information  needed  for  such  decisions.  Probabilistic  Simple 
Temporal  Problems  (PSTPs),  first  presented  in  (Tsamardi- 
nos  2002),  are  an  extension  of  STP-u  that  includes  such  in¬ 
formation  in  the  temporal  plan. 


Probabilistic  Simple  Temporal  Problems 

We  begin  by  defining  PSTPs. 

Definition  6.  A  Probabilistic  Simple  Temporal  Problem 
PSTP  is  a  tuple  <  Vc^E^Vu^  Par^  V  >,  where: 

•  Vc  is  the  set  of  controllable  variables  with  real  values, 

•  Vu  is  SL  set  of  real  random  variables  (uncontrollable  vari¬ 
ables). 

•  £'  a  set  of  constraints  of  the  form  X  —  Y  <  b,  X^Y  G 
Vc  U  Vu,  and  &  G  3?  U  {  — co,  oo}. 

•  Par  is  a  function  Vu  Vq  that  specifies  for  each  un¬ 
controllable  its  controllable  parent, 

•  is  a  set  of  conditional  probability  density  functions 
(pdf)  px  (t)  for  each  X  G  Vu  providing  the  mass  of  prob¬ 
ability  of  X  occurring  t  time  units  after  Par{X). 

It  is  worth  noting  that  in  PSTPs,  each  uncontrollable  event 
has  a  single  parent,  which  must  be  a  controllable  event; 
thus,  function  Par  is  well-defined.  The  probability  func¬ 
tions  in  P  deserve  further  comment.  P  is  a  set  of  probabil¬ 
ity  distributions  pxit)  summarizing  expectations  about  the 
occurrence  of  each  uncontrollable  event  X.  More  precisely, 
given  px{t),  the  probability  of  X  occumng  T  time  units  or 
less  after  Par{X)  has  occurred  is  given  by  Px{t  <  T)  = 

YooPxit)dt. 

Implicit  in  our  definitions  is  that  the  probability  distribu¬ 
tion  of  occurence  of  X  with  time  does  not  depend  on  abso¬ 
lute  time  but  on  relative  time  from  the  moment  the  parent  of 
X  is  executed  (px{t)  is  time  invariant). 

As  an  example,  suppose  that  we  want  to  model  the  fact 
that  an  action  A  has  duration  normally  distributed  with 
mean  duration  of  half  an  hour  (300  Y  standard  de¬ 
viation  cr.  We  define  the  beginning  of  the  action  as  the 
controllable  Y  =  start  (A),  and  the  end  of  the  action  as 
the  uncontrollable  X  =  end{A),  for  which  px{t)  follows 
Normal{30^  5)  (normal  with  half  an  hour  mean  and  5  min¬ 
utes  standard  deviation).  Then  we  can  find  out  the  prob¬ 
ability  that  the  action  will  finish  within  in  40  minutes  af¬ 
ter  F  (i.e.,  after  we  started  the  action):  P{t  <  40)  = 

fYpx(t)dt  =  =  97.72%,  where  $(.j)  is  the 

integral  of  the  Normal(0,l)  at  point  z  f 

Let  us  compare  modelling  action  A  in  a  PSTP  with  mod¬ 
elling  the  same  action  in  an  STP-u.  In  an  STP-u  one  has 
to  come  up  with  reasonable  bounds  I  and  u  and  specify  that 
I  <  X  -  Y  <  u  is  SL  contingent  link  to  be  included  in  the 
plan.  In  comparison,  in  an  PSTP  the  same  constraint  is  rep¬ 
resented  by  specifying  that  the  parent  of  X  is  F  and  that 
X  will  occur  t  time  units  after  F  where  t  follows  a  normal 
probability  distribution  with  mean  30^  and  standard  devia¬ 
tion  5^ 

Given  a  PSTP,  our  goal  is  to  assess  the  probability  that 
all  its  execution  constraints  E  will  be  satisfied  during  execu¬ 
tion.  More  specifically,  we  would  like  to  calculate  the  prob¬ 
ability  of  the  plan  being  legally  executed  under  an  optimal 
execution  strategy.  The  reason  for  doing  this  is  to  provide 

^This  integral  cannot  be  solved  analytically  but  is  typically 
computed  numerically  or  provided  in  a  table  form. 
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guidance  to  the  planning  process:  we  want  to  know  whether 
the  current  plan  is  “good  enough”,  i.e.,  likely  enough  to  suc¬ 
ceed,  or  whether,  instead,  further  effort  should  be  put  into 
looking  for  a  better  plan, 

(Tsamardinos  2002)  shows  how  to  find  an  optimal  execu¬ 
tion  strategy  for  PSTPs  under  certain  conditions.  However, 
the  basic  approach  there  is  a  static  one,  i.e.,  one  that  cor¬ 
responds  to  strong  controllability  in  STP-u’s,  To  compute 
the  equivalent  of  a  dynamic  execution  strategy,  it  is  neces¬ 
sary  to  iterate  the  process  of  finding  an  optimal  PSTP  execu¬ 
tion  strategy  whenever  an  observation  of  an  uncontrollable 
occurs.  Computing  the  exact  probability  of  success  of  this 
overall  dynamic  strategy  is  difficult.  However,  we  do  know 
how  to  compute  bounds  on  the  probability  of  success,  and 
that  is  what  we  focus  on  in  the  remainder  of  this  paper.  In 
the  next  section,  we  show  how  to  compute  an  upper  bound 
on  the  probability  of  success.  This  bound  can  be  used  by  a 
system  to  reject  a  plan,  if  it  is  too  low.  In  the  following  sec¬ 
tions,  we  describe  how  to  compute  a  lower  bound,  by  con¬ 
verting  a  PSTP  to  an  STP-u  and  then  tighening  the  latter’s 
bounds  until  it  becomes  dynamically  controllable. 

Bounding  the  Probability  of  Legal  Execution 
from  Above 

Suppose  that  an  uncontrollable  variable  X  with  parent  Y 
occurs  t  time  units  after  Y,  i.e.,  X  —  Y  =  T  If  there  is 
no  solution  to  the  constraints  in  E  that  admits  a  value  t  for 
the  difference  X  —  Y  then  the  probability  of  completing  the 
execution  in  a  way  that  respects  the  constraints  is  zero. 

As  we  mentioned  earlier,  the  distance  between  Y  and  X 
is  the  minimum  number  dyx  foi'  which  X  —  Y  <  dyx 
holds  in  all  solutions.  Similarly,  Y  —  X  <  dxy  holds  in  all 
solutions.  These  inequalities  together  imply  that  —dxy  < 
X  —  Y  <  dyx  any  legal  execution.  Therefore,  with 
probability  px{t)  for  t  outside  the  interval  [—dxy^  dyx]  a 
legal  execution  cannot  be  achieved.  Equivalently,  a  legal 
execution  can  be  achieved  with  probability  density  at  most 
Px{t)  for  t  within  the  interval  [—dxyjdyx]^ 

Assuming  all  uncontrollable  events  are  independent  of 
each  other,  and  using  Success  to  denote  the  event  of  a  legal 
execution  occuring,  then: 

P{Success)  <  JJ  Px{t  ^  [-dxYjdyx]) 

XcVuX=Parix) 

=  E  Px{-dxY  <t<dYx) 

XeVu,Y=Par(X) 

The  distances  dxY  can  be  determined  with  a  polyno¬ 
mial  all-pairs  shortest  path  algorithm.  The  calculation  of 
the  probabilities  in  the  product  depends  on  the  exact  density 
functions.  As  an  example,  if  px{i)  is  uniform  in  the  inter- 

val  [a, 6],  then  Px{-dxY  <  t  <  dyx)  =  = 

dvx+rfxv  )  assuming  [-dxY,dYx]  Q  [o,6]- 

As  an  example  consider  the  PSTP  in  Figure  1 .  The  dot¬ 
ted  edges  represent  temporal  constraints.  Specifically,  each 
edge  A  ^  B  annotated  with  the  interval  [/,  u]  represents  the 


Figure  1 :  Example  for  calculating  the  upper  bound. 


X-Y=-l 


Figure  2:  The  polytope  defined  by  the  constraints  provides  a 
tighter  upper  bound. 

two  inequality  constraints  I  <  B  —  A  <  u.  There  are  three 
such  constraints  (six  single  inequality  constraints): 

1<Y -TR<1 

S<  Z  -  TR<  10 

-1<Z -X  <2 
We  can  rewrite  these  as: 

-1  < Ti^-y  <  -1 

S<  Z  -  TR<  10 

-2<X  -  Z  <1 

By  adding  them  up  we  infer  that  5  <  X  —  Y  <10.  This  in¬ 
ferred  constraint  is  depicted  in  the  figure  with  the  annotation 
[5, 10]  below  the  solid  line.  It  is  equivalent  to  calculating  the 
distances  between  X  and  Y:  dxY  =  —5  and  dyx  =  10. 

The  solid  link  denotes  the  fact  that  X  is  an  uncontrollable 
variable  with  parent  Y  and  the  interval  on  top  of  this  edge 
shows  the  form  of  this  dependency:  X  will  occur  some  time 
within  the  interval  [5, 15]  after  Y  has  been  executed  with 
unifonn  distribution.  For  example  Y  may  be  the  beginning 
of  an  action  with  duration  between  5  and  15  and  X  the  end 
of  this  action. 

As  calculated,  there  is  a  solution  to  the  constraints  only 
if  5  <  A  —  y  <10.  Thus,  with  at  most  probability  of 
Pxi^  <  t  <  10)  =  =  I  the  PSTP  can  be  executed 

successfully. 

The  upper  bound  calculated  with  the  above  method  may 
not  be  tight  in  general.  We  now  discuss  ideas  why  this  is  the 
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case  and  how  to  find  tighter  bounds.  Consider  a  PSTP  with 
two  uncontrollables  X  and  Y  that  both  occur  with  uniform 
probability  in  [1,3]  after  the  time  reference  point  TR,  Let 
us  assume  that  the  only  constraints  in  this  PSTP  are  —  1  < 
X-randX-r  <  1. 

Figure  2  shows  the  space  of  legal  executions.  The  ar-axis 
is  the  time  of  occurence  of  X  and  similarly  for  Y.  Since 
TR  =  0  the  set  of  legal  execution  is  the  area  of  the  rectan¬ 
gle  bounded  by  the  lines  X  —  Y  =  — 1  and  X  —  Y  =  1. 
Calculating  the  bounds  with  the  method  that  we  provided 
yields:  Px{—dx,TR  ^  ^  ^  dTR^x)PY{—dYTR  ^  ^  ^ 
dTR,Y)  —  Px{—oo  <t<  oo)Py{—oo  <t<  oo)  =  1. 

A  tighter  bound  on  the  probability  in  this  example  would 
be  the  area  of  the  rectangle  bounded  by  the  two  constraints. 
In  general,  a  tighter  bound  could  be  obtained  by  calculating 
the  mass  of  probability  contained  in  the  polytope  defined  by 
the  constraints. 

The  mass  of  probability  of  this  polytope  is  still  only  an 
upper  bound  on  the  probability  of  correct  execution:  even 
though  for  every  point  in  this  polytope  (i.e.,  execution)  the 
constraints  are  satisfied,  that  does  not  mean  that  an  agent 
will  dynamically  be  able  to  construct  this  solution,  unless  it 
is  clairvoyant  in  the  general  case. 

Bounding  the  Probability  of  Legal  Execution 
from  Below 

Let  <  Vc->E^Vu->  Par,  P  >  be  a  PSTP  and  suppose  that  we 
are  given  intervals  [IxjRx]  for  oach  uncontrollable  variable 
X  with  parent  Y.  The  PSTP  and  the  intervals  can  be  seen  as 
corresponding  to  an  STP-u  with  the  same  controllable  and 
uncontrollable  variables,  same  constraints,  and  contingent 
links  lx  ^  X  —  y  <  ux  for  each  uncontrollable  variable. 

If  this  STP-u  is  dynamically  controllable  then  there  ex¬ 
ists  an  execution  strategy  for  legally  executing  the  STP-u 
no  matter  when  the  uncontrollables  occur  within  these  inter¬ 
vals.  Thus,  in  all  such  cases  where  the  uncontrollables  oc¬ 
cur  within  these  bounds  an  agent  can  execute  the  plan  with 
probability  one,  provided  it  follows  the  execution  strategy 
returned  by  the  STP-u  controllability  algorithm.  The  prob¬ 
ability  of  all  such  cases  is  thus  a  lower  bound  on  the  proba¬ 
bility  of  a  legal  execution.  Thus: 

P{Success)  >  JJ  Px{lx  <  ^  <  Ux) 
xevu 

Unlike  the  upper  bound  that  we  provided  in  the  previ¬ 
ous  section,  in  the  lower  bound  case  the  bounding  intervals 
[Ix^ux]  cannot  be  easily  computed  (because  it  is  required 
that  the  corresponding  STP-u  is  controllable).  The  looser 
these  intervals  the  tighter  the  lower  bound  will  be.  To  find 
the  tightest  bound  possible  one  needs  to  solve  the  following 
optimization  problem: 

Definition  7.  Lower  Bound  Problem. 

Given  PSTP  <  Vc,E,Vu,  Par,  V  >  : 

Maximize  JJxeVu  Px{lx  <t<  ux) 
subject  to: 

<  Vc,E,Vu,C  >  being  dynamically  controllable 


where  C  is  the  set  of  contingent  links 

{lx<X-Y  <ux\X^Vu^Y^  Par{X)} 
and  decision  variables: 

lx,ux^  for  each  X  ^Vu 

Unfortunately,  it  is  difficult  to  directly  apply  typical  con¬ 
straint  optimization  techniques  such  as  gradient  descent  or 
Newton’s  Method  on  this  problem.  This  is  because  such 
methods  require  expressing  the  feasible  set  as  the  decision 
variable  vectors  that  satisfy  a  set  of  equality  or  inequality 
constraints.  In  the  lower  bound  problem  the  feasible  region 
is  the  set  of  decision  variable  vectors  that  satisfy  the  single 
constraint  that  the  corresponding  STP-u  is  dynamically  con¬ 
trollable. 

In  the  following  two  sections  we  sketch  candidate  algo¬ 
rithms  that  approximate  the  optimal  solution  to  the  lower 
bound  problem.  We  then  return  to  the  formulation  of  the 
problem  as  an  optimization  problem  and  suggest  ways  to 
convert  it  to  a  form  suitable  for  classical  optimization  tech¬ 
niques  in  a  way  that  approximates  the  problem  we  are  trying 
to  solve. 

Binary  Search  for  Loosest  Bounds 

In  our  first  algorithm,  we  perform  binary  search  for  the 
bounds  on  the  uncontrollable  intervals  that  are  as  loose  as 
possible  while  still  guaranteeing  dynamic  controllability. 
The  basic  algorithm  is  as  follows: 

1.  Given  PSTP  <  Vc-,  E,  Vu,  Par,  V  >,  construct  a  corre¬ 
sponding  STP-u  S  <  Vl.,E\Vlj,C  >  where  =  Vc, 
VIj  =  Vu,  E'  =  E,mdC  =  lx  <  X  -  Par(X)  <  ux  for 
each  X  e  Vc,  where  lx  and  wx  are  initialized  to  include 
most  of  the  mass  of  probability  of  px-  (For  example,  lx 
might  be  the  mean  minus  three  standard  deviations,  while 
Ux  might  be  the  mean  plus  three  standard  deviations.) 

2.  Let  e  be  a  small  threshold  value,  and  Vi  E  —  1. 

3.  While  (S  is  not  dynamically  consistent)  and  {E  >  e) 

4.  Begin 

5.  If  S  is  not  dynamically  controllable 

6.  F  =  F/2 

7.  Reduce  all  contingent  edges  in  S  by  a  factor  of  F 

8.  Else 

9.  F  =  3F/2 

10.  Increase  all  contingent  edges  in  S  by  a  factor  of  F 

11.  End  If 

12.  End  While 

13.  Return  S. 

Note  that  this  algorithm  assumes  that  the  underlying  STP- 
u  can  eventually  be  made  consistent  by  shrinking  the  bounds 
on  the  uncontrollable  events  far  enough:  i.e.,  if  the  time 
points  of  the  uncontrollables  could  be  pinned  down,  the  net¬ 
work  would  be  executable.  Also,  we  have  made  an  arbitaiy 
decision  about  the  rate  at  which  we  modify  the  size  of  the  in¬ 
tervals,  reducing  them  by  a  half  when  the  network  is  not  dy¬ 
namically  controllable,  and  increasing  them  by  a  half  when 
it  is.  To  achieve  faster  convergence,  we  may  want  to  vary 
these  values. 

This  basic  algorithm  can  be  improved  in  several  ways, 
Eirst,  when  an  STP-u  is  not  dynamically  uncontrollable,  this 
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Figure  3:  Two  uncontrollable  events  with  different  distribu¬ 
tions. 


^  1  [u  v]  - ^  Contingent  Link 

[p,q]  "  -  '  ’ 

" '  - ^  Requirement  Link 

V) 

Figure  4:  Triangular  Networks  (Morris,  Muscettola,  &  Vidal 

2001). 


may  be  due  only  to  some,  and  not  all,  of  the  uncontrollable 
events.  It  may  be  possible  to  identify  which  uncontrollable 
events  are  to  blame  while  running  the  STP-u  controllabil¬ 
ity  algorithm  and  then  to  modify  the  above  algorithm  so 
that  only  the  edges  incident  upon  culpable  events  are  re¬ 
duced.  Second,  the  above  algorithm  does  not  take  account 
of  the  fact  that  the  PSTP  explicitly  models  the  probability 
distribution  associated  with  each  uncontrollable  event.  In¬ 
stead,  it  reduces  the  time  intervals  associated  with  all  events 
equally.  An  improved  appropriate  extension  would  be  to  re¬ 
duce  the  bounds  proportionally  to  the  probability  mass  as¬ 
sociated  with  each  interval.  For  example,  if  one  contingent 
interval  has  a  distribution  with  very  wide  variance,  while 
another  has  a  much  steeper  distribution  with  narrower  vari¬ 
ance,  we  would  prefer  to  place  tighter  bounds  on  the  first- 
or,  put  otherwise,  shrink  the  first  interval  more-than  the  sec¬ 
ond,  because  that  would  result  in  less  reduction  in  the  overall 
probability  mass  of  the  uncontrollable  events  modeled.  (See 
Figure  3.) 

Iterative  Tightening 

The  Binary  Search  approach  employs  the  dynamical  control¬ 
lability  algorithm  as  a  black  box.  The  Iterative  Tightening 
approach  on  the  other  hand  modifies  the  dynamic  controlla¬ 
bility  algorithm. 

The  Iterative  Tightening  first  converts  the  PSTP  to  an 
STP-u  by  calculating  loose  bounds  for  the  uncontrollables 
in  a  way  that  contain  most  (or  all  if  possible)  of  the  mass  of 
probability  (exactly  as  the  Binary  Search  algorithm).  Then, 
it  runs  a  modified  dynamic  controllability  algorithm:  instead 
of  stopping  as  soon  as  it  is  discovered  that  the  STP-u  is  not 
controllable,  the  algorithm  relaxes  the  problem  (by  tighten¬ 
ing  the  bounds)  and  continues. 

The  dynamic  controllability  algorithm  checks  each  triplet 
of  variables  A^B  and  C,  where  C  is  uncontrollable  (as 
shown  in  Figure  4.  The  constraints  (requirement  links) 
A  ^  B  and  B  ^  C  may  be  explicit  or  implicit  con¬ 
straints.  The  algorithm  then  imposes  a  set  of  additional  con¬ 
straints  that  ensure  the  exi stance  of  an  execution  strategy. 
If  the  propagation  of  these  constraints  does  not  result  in  a 
“squeeze”  of  the  contingent  link,  the  STP-u  is  controllable. 

The  Iterative  Tightening  algorithm  employs  the  same 
strategy.  It  selects  a  triangle  of  variables  to  work  on  and 
imposes  the  constraints  determined  by  the  controllability  al¬ 
gorithm,  However,  if  the  propagation  of  these  constraints 
squeeze  any  other  contingent  link,  instead  of  stopping,  the 
algorithm  tightens  the  contingent  link  to  these  new  bounds. 
Obviously,  this  algorithm  will  not  return  an  execution  strat¬ 


egy  that  works  for  all  possible  occurences  of  the  uncontrol¬ 
lables  of  the  original  bounds,  but  only  for  the  final  tightened 
bounds. 

The  algorithm  is  as  follows: 

1.  Given  PSTP  <  Vc,E,Vu,  Par,  V  >,  construct  a  corre¬ 
sponding  STP-u  S  <  VI.,  E^,VIj,C  >  where  =  Ffc, 
VI  =  Vu,  E^  =  E,  and  C  =  lx  <X-  Far(X)  <  ux 
for  each  X  E  Vu,  where  lx  and  ux  are  initialized  to 
include  most  of  the  mass  of  probability  of  px^ 

2.  Non-deterministically  CHOOSE  a  triangle  of  variables 
A,  B,  C  where  C  is  uncontrollable. 

3.  Impose  the  constraints  determined  by  the  dynamic  con¬ 
trollability  algorithm, 

4.  Propagate  the  constraints  as  in  the  controllability  algo¬ 
rithm,  but  allow  requirement  links  to  be  squeezed, 

5.  Repeat  until  the  lower  bound  that  corresponds  to  the  cur¬ 
rent  STP-u  is  high  enough,  or  the  last  constraint  propaga¬ 
tion  did  not  change  the  STP-u. 

In  Iterative  Tightening  the  order  of  consideration  of  each 
triangle  matters.  When  a  triangle  A,  B,  C  and  a  contingent 
link  A  ^  C  is  selected  and  appropriate  constraints  are  im¬ 
posed  to  ensure  controllability,  essentially  the  algorithm  cre¬ 
ates  an  execution  strategy  that  works  for  all  cases  where  C 
occurs  within  the  current  bounds  given  for  A  ^  C.  Propa¬ 
gation  of  these  constraints  may  require  that  other  uncontrol¬ 
lables  occur  within  a  tighter  interval  to  allow  for  this  strategy 
to  work. 

For  example,  suppose  that  there  are  two  contingent  links 
Y  ^  X  and  A  ^  B.  Selecting  a  triangle  that  involves  the 
first  one  may  cause  the  bounds  on  the  second  one  to  shrink 
considerably  in  order  to  allow  the  execution  strategy  to  work 
with  all  possible  occurences  of  X,  If  B's  probability  distri¬ 
bution  has  heavy  tails,  that  means  that  a  significant  mass 
or  probability  will  be  excluded  from  the  calculation  of  the 
lower  bound.  If  instead  the  second  triangle  is  selected  first, 
its  bounds  will  not  be  tightened  but  may  cause  the  bounds 
on  the  first  link  to  shrink.  If  however,  the  distribution  of  X 
has  smaller  variance,  then  shrinking  the  bounds  will  not  ex¬ 
clude  as  much  probability  mass  and  the  algorithm  will  return 
a  tigher  lower  bound. 

Possible  variants  of  the  algorithm  include  a  backtracking 
search  where  different  choices  of  triangles  are  made  in  a 
search  for  the  STP-u  that  provides  the  tighest  lower  bound 
on  the  probability  of  successful  execution. 
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Non-Linear  Optimization  Solutions 

In  this  section,  we  explore  the  possibility  of  solving  the 
lower  bound  problem  with  optimization  methods.  While  the 
objective  function  is  suitable  for  typical  optimization  meth¬ 
ods,  the  constraints  are  not.  We  now  attempt  to  cast  the 
constraint  of  the  resulting  STP-u  being  dynamically  control¬ 
lable,  as  a  set  of  inequalities,  which  would  allow  non-linear 
optimization  techniques  to  be  used. 

Consider  the  triangle  of  variables  and  edges  of  Figure  4, 
where  the  edge  A  ^  (7  is  a  contingent  link.  According  to 
(Morris,  Muscettola,  &  Vidal  2001)  the  triangle  is  dynam¬ 
ically  controllable  if  any  of  the  following  three  conditions 
hold: 

1 .  V  <0,  and  the  triangle  is  pseudo-controllable, 

2.  u  >  Q,  B  —  A  C  [y  —  V ^  X  —  u],  and  the  triangle  is  pseudo- 
controllable. 

3.  V  >  0,  w  <  0,  y  —  V  <  X,  and  the  triangle  is  pseudo- 
controllable. 

Pseudo-controllability  denotes  the  fact  that  the  bounds 
[x,y]  are  not  “squeezed”  by  the  contraints  of  the  triangle, 
or  in  other  words  that  [x,  y]  C  [—dcA^  dAc]-  Recall  that  the 
dynamic  controllability  algorithm  determines  whether  there 
is  a  way  to  execute  the  plan  no  matter  when  the  uncontrol- 
lables  occur.  The  interval  [—dcA.dAc]  is  the  interval  dic¬ 
tated  by  the  constraints  in  the  plan:  any  time  within  that  in¬ 
terval  participates  in  at  least  one  solution  of  the  constraints. 
The  interval  [x^  y]  is  derived  from  the  contingent  link  and  is 
a  constraint  on  Nature.  Thus,  if  there  are  values  of  [x^  y]  that 
do  not  participate  in  any  solution  of  the  constraints.  Nature 
may  select  one  of  these  values  forbidding  the  completion  of 
the  plan  in  a  way  that  satisfies  the  constraints. 

Apart  from  the  three  cases  above  for  when  a  triangle  is 
dynamically  controllable  there  is  a  fourth  case.  Thus,  the 
above  three  cases  together  are  sufficient  but  not  necessary 
conditions.  The  fourth  case  involves  accepting  a  ternary  and 
disjunctive  constraint  that  is  called  a  wait,  which  we  will 
ignore  for  the  moment. 

An  STP-u  is  dynamically  controllable  if  all  such  triangles 
in  the  network  are  dynamically  controllable.  These  three 
cases  direct  the  design  of  the  our  algorithm. 

1.  GiwQn  SiFSTF  <Vc,E,Vu,Par,V  >. 

2.  Define  a  non-linear  optimization  problem  with  decision 
variables  Xi,yi  for  every  uncontrollable,  objective  func- 

Yli  Pci^i  ^Vi)^  inequality  constraints  S  as 
defined  below. 

3.  Initialize  S  ^  E. 

4.  For  each  triple  of  variables  Ai^Bi^Ci  as  in  Figure  4, 
where  Ci  is  uncontrollable: 

•  lfvi<  0,  then  no  extra  constraint  needs  to  be  added, 

•  If  Ui  >  0,  then  S  ^  S  A  {Bi  —  A^  C  [yi  —  Vi^Xi—Ui]} 

•  Else,  S  ^  S  U  {yi  —  Vi  <  Xi}, 

5.  Solve  the  optimization  problem. 

The  solution  to  the  optimization  problem  will  return  a  set 
of  values  for  the  decision  variables  for  which  all  the  con¬ 
straints  are  satisfied.  By  construction,  satisfying  all  these 


constraints  implies  that  the  corresponding  STP-u  is  dynam¬ 
ically  controllable.  This  is  because  each  such  triangle  falls 
into  one  of  the  three  cases  above. 

In  addition,  in  any  solution  of  the  optimization  problem 
the  triangles  are  pseudo-controllable.  This  is  because  any 
bounds  X,  y  selected  by  the  optimization  for  a  contingent 
link  A  ^  (7  are  as  squeezed  as  possible:  y  <  dAc  because 
if  ^  >  dAc  then  y  is  outside  the  feasible  set  imposed  by  the 
constraints  of  the  optimization  problem. 

Intuitively,  the  algorithm  is  free  to  select  any  x,  y  bounds 
on  contingent  links  and  impose  any  constraints  on  Nature 
desired.  Of  course  Nature  may  not  observe  these  constraints 
but  we  can  calculate  the  probability  that  she  will  and  obtain 
the  desired  lower  bound. 

Notice  that  since  the  three  cases  are  sufficient  but  not  nec¬ 
essary  it  is  conceivable  that  this  is  not  the  tightest  lower 
bound  on  the  probability  that  can  be  achieved  using  this  kind 
of  approach  (i.e.,  by  translating  to  an  STP-u).  Specifically, 
there  is  a  fourth  case  that  we  omitted  from  consideration:  it 
involves  a  disjunctive  and  ternaiy  constraint  called  a  wait  on 
(7.  For  example  wait  <  (7, 5  >  means  that  one  should  wait 
to  execute  B  until  5  time  units  have  passed  after  A  has  been 
executed  or  C  has  been  observed.  A  non-linear  optimization 
algorithm  that  takes  into  consideration  this  case  may  be  able 
to  further  increase  the  bounds  [x,  y]  to  include  more  mass  of 
probability.  It  is  currently  unknown  how  significant  is  this 
case  in  practice  and  how  much  looser  than  optimal  is  a  lower 
bound  that  is  achieved  by  ignoring  this  case. 

We  now  consider  a  specific  class  of  probability  density 
functions  and  the  corresponding  optimization  problems  they 
give  rise  to. 


Optimization  for  Uniform  Distributions 

Let  us  denote  with  pi  the  pdf  of  the  rth  uncontrollable  vari¬ 
able  and  suppose  that  all  probability  distributions  are  uni¬ 
form,  that  is  pi{t)  =  ,  when  t  G  [a^,  bi]  and  zero  out¬ 

side  this  interval. 

If  p{t)  is  uniform  in  [a,  6],  then 

for  X  <  ^.  In  Figure  5  this  is  justified  pictorially  where  the 
probability  density  of  a  uniform  pi  within  the  bounds  [a,  b] 
is  shown.  P{x  <  t  <  y)  is  the  area  above  the  intersection 
of  [a,b]  and  [x,y]. 

Instead  of  maximizing  the  actual  probability  of  successful 
execution,  we  can  maximize  its  logarithm. 

max  log  n  Pi{xi  <t<yi) 

i 

which  is  equal  to 

max  E  \ogPi{xi  <t<yi) 


By  utilizing  the  fact  that  P^s  are  uniform,  this  is  equivalent 
to 


max  E  log 

i 


bi  ai 
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X 


p(t) 


Figure  5:  The  mass  of  probability  of  a  random  variable  uni¬ 
formly  distributed  within  [x,  y]  occuring  within  [a,  h]  is  the 
area  above  the  intersection  of  the  intervals. 


where  ai  —  max(xi,a)  and  —  mm{yi,b).  In  turn,  this 
gives: 


max(^  log(6  -  ^og{bi  -  ai)) 

i  i 

The  last  sum  is  a  constant  term  and  can  be  dropped  from 
the  objective  function  during  optimization  (but  is  required  to 
compute  the  final  bound  on  the  probability).  So  the  objective 
function  becomes 

max  E  log(6 


or  equivalently 


min  -y^log(^i  -  ai) 

i 

The  constraints  of  this  optimization  problem  are  given  by 
Steps  3  and  4  in  the  algorithm  of  the  previous  section.  That 
is,  they  are  the  difference  constraints  among  the  PSTP  vari¬ 
ables  (controllable  and  uncontrollable  ones)  union  the  con¬ 
straints  required  to  guarantee  the  resulting  STP-u  is  dynam¬ 
ically  controllable.  In  addition  to  these  constraints  however, 
we  need  to  add  that  ai  —  max(xi,  a^),  =  min(^^,  hi),  and 

that  Xi<yi, 

The  max  and  min  functions  present  problems  to  most  op¬ 
timization  algorithms.  Fortunately,  in  this  case  we  can  sub¬ 
stitute  —  min(y^,  hi)  with  the  constraints  <  yi, 

and  ai  —  max(x^,ai)  with  ai  >  ai,  and  ai  >  Xi,  This 
is  because,  in  order  to  maximize  the  objective  function  the 
^i  should  be  as  large  as  possible;  so  the  optimization  en¬ 
gine  will  increase  the  ^i  until  at  least  one  of  the  equalities 


H  is  semidef- 


=  bi^^i  =  yi  holds,  in  which  case  ^i  —  min(y^,6).  A 
similar  argument  holds  for  ai. 

The  feasible  region  defined  by  these  inequality  constraints 
is  convex.  Additionally,  the  objective  function  is  twice 
differentiable  everywhere  except  from  the  boundaiy  where 
cTi  =  6-  So  the  objective  function  is  twice  differentiable  in 
the  interior  of  the  feasible  region. 

Let  us  calculate  the  second  derivative  of  each  term  in  the 
sum.  Define /(C,o-)  =  -  \og{^-a).  Then,  V/(C,  cr)  = 
[-(^-cr)-i,(^-cr)-i]  =  [-a, a],  fora  =  (^-cr)-^The 

Hessianisif  = 

inite  positive  because  the  eigenvalues  are  non-negative.  The 
eigenvalues  A  solve  det ( AI  —  V^/(^,cr))  =  A^—  2o? A  =  0, 
i.e.,  A  =  0  or  A  =  2a^  >  0  for  (j  <  ^.  Thus,  function  /  de¬ 
fined  on  a  convex  set  (when  a  <  ^)  has  a  positive  semidef- 
inite  V^/  in  the  interior  and  thus  is  convex  (provided  the 
interior  is  non-empty).  The  objective  function,  as  a  sum  of 
such  convex  functions  is  also  convex. 

Convex  optimization  problems  have  a  unique  minimum 
and  in  general,  are  relatively  easily  solved  with  modern  op¬ 
timization  software.  We  are  currently  considering  other  fam¬ 
ilies  of  probability  distributions  such  as  exponential  or  nor¬ 
mal  distributions. 


Discussion 

The  recent  literature  on  planning  has  shown  a  growing  inter¬ 
est  in  handling  more  and  more  realistic  problems,  and  along 
with  that  has  come  a  concern  with  various  types  of  uncer¬ 
tainty,  In  this  paper,  our  focus  has  been  on  temporal  uncer¬ 
tainty:  uncertainty  about  the  time  at  which  certain  exoge¬ 
nous,  or  “uncontrollable”  events  will  occur.  Significantly, 
domain  experts  typically  know  more  about  such  events  than 
just  the  interval  of  time  during  which  they  will  occur-they 
often  know  a  probability  distribution  over  the  interval  of  oc¬ 
currence.  Yet  the  most  successful  formalism  for  planning 
with  temporal  uncertainty,  the  STP-u ’s,  don't  allow  one  to 
exploit  that  knowledge.  Instead,  the  domain  expert  must 
specify  fixed  bounds  on  the  time  during  which  each  uncon¬ 
trollable  event  must  happen.  If  he  sets  the  bounds  too  nar¬ 
rowly,  he  may  produce  a  plan  that  is  dynamically  control¬ 
lable,  but  that  nonetheless  fails,  because  the  uncontrollable 
event  occurs  outside  the  modeled  time.  If  he  sets  them  too 
widely,  he  may  produce  a  plan  that  is  not  dynamically  con¬ 
trollable.  And  execution  strategies  only  exist  for  dynami¬ 
cally  controllable  plans;  if  an  STP-u  is  not  dynamically  con¬ 
trollable,  there  is  no  effective  means  of  deciding  when  to 
execute  the  controllable  actions  in  it. 

This  poses  a  real  problem  for  the  designer  of  a  planner 
dealing  with  temporal  uncertainty.  It  is  difficult  to  know 
how  to  set  the  bounds  on  the  uncontrollable  events;  and  if 
the  bounds  are  set  too  widely,  it  is  impossible  to  assess  the 
probability  that  the  plan  can  nonetheless  be  legally  executed, 
and  thus,  impossible  to  make  a  principled  decision  about 
whether  to  adopt  this  plan  or  whether  to  search  further  for 
an  alternative. 

What  we  would  like  to  do  is  to  enable  the  domain  ex¬ 
pert  to  specify  bounds  on  the  uncontrollable  events  that  are 
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“as  wide  as  possible”:  by  this  we  mean  that  they  maximize 
the  probability  that  the  events  will  in  fact  occur  during  the 
modeled  bounds,  subject  to  the  constraint  that  the  network 
is  dynamically  controllable.  When  the  bounds  are  set  in  this 
fashion,  there  are  two  results:  first,  we  have  an  evaluation  of 
the  probability  that  the  plan  can  be  legally  executed,  which 
can  be  used  to  decide  whether  to  accept  or  reject  it,  and  sec¬ 
ond,  if  it  is  accepted,  then  the  network  with  the  bounds  thus 
set  can  serve  as  the  basis  of  a  legal  execution  strategy. 
Because  STP-u’s  do  not  include  information  about  the 
probability  distribution  of  the  timing  of  uncontrollable 
events,  we  presented  a  generalization  of  them,  called  Proba¬ 
bilistic  Simple  Temporal  Problems  (PSTPs),  Unfortunately, 
given  an  PSTP,  it  is  difficult  to  compute  an  exact  probability 
of  legal  execution.  What  we  can  do,  however,  is  bound  the 
probability,  both  from  above  and  below.  The  upper  bound 
simply  provides  a  way  of  rejecting  a  plan  if  it  is  not  below  a 
given  threshold.  The  lower  bound  is  arguably  more  interest¬ 
ing,  as  it  is  not  only  what  gives  a  way  of  accepting  a  plan, 
when  it  is  above  a  threshold,  but  is  also  what  allows  one  to 
approximate  the  widest  possible  bounds. 

We  presented  three  alternative  algorithms  for  approximat¬ 
ing  the  widest  possible  bounds.  The  first  performs  binary 
search  for  a  value  v  that  represents  the  minimal  proportion 
by  which  all  the  uncontrollable  intervals  need  to  be  reduced 
to  achieve  a  dynamic  controllability.  The  second  runs  the 
dynamic  controllability  algorithm  with  the  modification  that 
it  does  not  exit  as  soon  as  controllability  is  deemed  impos¬ 
sible.  Instead,  it  shrinks  the  intervals  appropriately,  relax¬ 
ing  the  initial  problem,  until  controllability  is  achieved.  The 
third  algorithm  takes  a  very  different  approach,  attempting 
to  reduce  the  problem  to  one  of  non-linear  optimization.  It 
approximates  the  set  of  controllable  STP-u’s  with  a  set  of 
inequality  constraints.  The  next  major  step  in  this  work  it  to 
implement  these  three  algorithms  and  conduct  both  experi¬ 
mental  analyses  of  their  performance  in  terms  of  computa¬ 
tional  efficiency  and  quality  of  lower  bounds  returned.  Ad¬ 
ditionally,  it  will  be  important  to  integrate  work  on  temporal 
uncertainty  of  the  kind  described  in  this  paper  with  work  on 
causal  uncertainty,  such  as  that  discussed  in  (Tsamardinos, 
Vidal,  &  Pollack  2003). 
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Abstract.  Many  systems  are  designed  to  perform  both  planning  and  execution: 
they  include  a  plan  deliberation  component  to  produce  plans  that  are  then  dis¬ 
patched  to  an  execution  component,  or  executive,  which  is  responsible  for  the 
performance  of  the  actions  in  the  plan.  When  the  plans  have  temporal  con¬ 
straints,  dispatch  may  be  non-trivial,  and  the  system  may  include  a  distinct  dis¬ 
patcher,  which  is  responsible  for  ensuring  that  all  temporal  constraints  are  satis¬ 
fied  by  the  executive.  Prior  work  on  dispatch  has  focused  on  plans  that  can  be 
expressed  as  Simple  Temporal  Problems  (STPs).  In  this  paper,  we  sketch  a  dis¬ 
patch  algorithm  that  is  applicable  to  a  much  broader  set  of  plans,  namely  those 
that  can  be  cast  as  Disjunctive  Temporal  Problems  (DTPs),  and  we  identify  four 
key  properties  of  the  algorithm. 


1  Introduction 

Many  systems  are  designed  to  perform  both  planning  and  execution:  they  include  a 
plan  deliberation  component  to  produce  plans  that  are  then  dispatched  to  an  execution 
component,  or  executive,  which  is  responsible  for  the  performance  of  the  actions  in 
the  plan.  When  the  plans  have  temporal  constraints,  dispatch  may  be  non-trivial,  and 
the  system  may  include  a  distinct  dispatcher,  which  is  responsible  for  ensuring  that  all 
temporal  constraints  are  satisfied  by  the  executive.  Prior  work  on  plan  dispatch  [1-3] 
has  focused  on  plans  that  can  be  represented  as  Simple  Temporal  Problems  (STP)  [4]. 
In  this  paper,  we  sketch  a  dispatch  algorithm  that  is  applicable  to  a  much  broader  set 
of  plans,  those  that  can  be  cast  as  Disjunctive  Temporal  Problems  (DTPs),  and  iden¬ 
tify  four  key  properties  of  the  algorithm. 


2  Disjunctive  Temporal  Problems 

Definition.  A  Disjunctive  Temporal  Problem  (DTP)  is  a  constraint  satisfaction 
problem  <V,  C>,  where  V  is  a  set  of  variables  (or  nodes)  whose  domains  are  the  real 
numbers,  and  C  is  a  set  of  disjunctive  constraints  of  the  form  Q.*  li  <  Xi  -  yi  <  Ui  v 
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. . .  V  ln<  Xn-  Un,  such  that  for  \  <i  <71,  Xi  and  yi  are  both  members  of  V,  and  It ,  Ui 
are  real  numbers.  An  exact  solution  to  a  DTP  is  an  assignment  to  each  variable  in  V 
satistying  all  the  constraints  in  C.  If  a  DTP  has  at  least  one  exact  solution,  it  is  consis¬ 
tent. 

A  DTP  can  be  seen  as  encoding  a  collection  of  alternative  Simple  Temporal  Prob¬ 
lems  (STPs).  To  see  this,  note  that  each  constraint  in  a  DTP  is  a  disjunction  of  one  or 
more  STP-style  inequalities.  Let  Cy  be  the  7-th  disjunct  of  the  /-the  constraint  of  the 
DTP.  If  we  select  one  disjunct  Cy  from  each  constraint  Q,  then  the  set  of  selected 
disjuncts  forms  an  STP,  which  we  will  call  a  component  STP  of  a  given  DTP.  It  is 
easy  to  see  that  a  DTP  D  is  consistent  if  and  only  if  it  contains  at  least  one  consistent 
component  STP.  Moreover,  any  solution  to  a  consistent  component  STP  of  D  is  also 
clearly  an  exact  solution  to  D  itself. 

Definition.  A(n  inexact)  solution  to  a  DTP  is  a  consistent  component  STP  of  it.  The 
solution  set  for  a  DTP  is  the  set  of  all  its  solutions. 

When  we  speak  of  a  solution  to  a  DTP,  we  shall  mean  an  inexact  solution.  Plans  can 
be  cast  as  DTPs  by  including  variables  for  the  start  and  end  points  of  each  action. 


3  A  Dispatch  Example 

Consider  a  very  simple  example  of  a  plan  with  three  actions,  P,  Q,  and  R.  (For  presen¬ 
tational  simplicity,  we  assume  each  action  is  instantaneous  and  thus  represented  by  a 
single  node).  P  must  occur  in  the  interval  [5,10]  and  Q  in  the  interval  [15,20];  P  and 
Q  must  be  separated  by  at  least  6  time  units;  and  R  must  be  performed  either  the  inter¬ 
val  [11,12]  or  [21,22].  The  plan  as  described  can  be  represented  as  the  following 
DTP:  {Cl.  5  <  P  -  77?  <  10  V  15  <  P  -  77?  <  20;  C2.  5  <  2  -  PP  <  10  v  15  <  2  - 
PP<20;  C?>.  6<P-Q<oow6<Q-P<oo,  C4.  11  <  P  -  PP  <  12  v  21  <  P - 
PP  <  22}.  (Note  that  PP,  the  time  reference  point,  denotes  an  arbitrary  starting 
point.)  This  DTP  has  four  (inexact)  solutions:  {  STPi:  Cjj,  C22,  C32,  C41;  STP 2:  Cjj, 
C22,  C32y  C42;  STP 3:  C12,  C21,  C31,  C41;  STP 4:  C12,  C21,  C31,  C42}. 

Definition:  An  STP  variable  v  is  enabled  if  and  only  if  all  the  events  that  are  con¬ 
strained  to  occur  before  it  have  already  been  executed.  A  DTP  variable  v  is  enabled  if 
and  only  if  it  has  a  consistent  component  STP  in  which  v  is  enabled. 

In  STPi,  both  P  and  P  are  initially  enabled,  while  in  STP3  and  STP4,  Q  is  initially 
enabled.  Hence,  all  three  actions  are  initially  enabled  for  the  DTP.  Enablement  is  a 
necessary  but  not  sufficient  condition  for  execution:  an  action  must  also  be  live,  in  the 
sense  that  the  temporal  constraints  pertaining  to  its  clock  time  of  execution  are  satis¬ 
fied.  In  the  current  example,  none  of  the  actions  are  initially  live.  The  first  action  to 
become  live  is  P,  at  time  5.  An  action  is  live  during  its  time  window. 
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Definition:  The  time  window  of  an  STP  variable  x  is  a  pair  {l,u\  such  that  /  <  x-TR 
<  u,  and  for  all  V,  u’  such  that  /’  <  x  -  TR  <  u\  V  <l  and  u  <  u\  Given  a  set  of 
consistent  component  STPs  for  a  DTP,  we  will  write  TW  {x,  i)  to  denote  the  time  win¬ 
dow  for  variable  v  in  the  such  STP.  The  upper  bound  of  a  time  window  [l,u}  for  v 
in  STP  i,  written  U(v,/),  is  u.  The  time  window  of  a  DTP  variable  x  is  TW  (v)=u^^ 
TW  (v,/),  where  S  is  the  solution  set  of  D. 

The  dispatcher  can  provide  information  about  when  actions  are  enabled  and  live  in 
an  Execution  Table  (ET).  This  is  a  list  of  ordered  pairs,  one  for  each  enabled  action. 
The  first  element  of  the  entry  specifies  the  action,  and  the  second  is  a  list  of  the  con¬ 
vex  intervals  in  that  element’s  time  window.  For  our  example,  then,  the  initial  ET 
would  be:  {<P,  {[5,10],  [15,20] }>,  <2,  {[5,10], [15,20] }}>,  <R, 

{ [1 1,12],[21,22]  }>}.  The  ET  summarizes  the  information  in  the  solution  STPs  so  that 
the  executive  does  not  have  to  handle  them  directly. 

The  ET  provides  information  about  what  actions  may  be  performed,  but  it  does  not 
provide  enough  information  for  the  executive  to  determine  what  actions  must  be  per¬ 
formed.  To  see  this,  note  that  the  ET  just  given  does  not  indicate  that  there  is  a  prob¬ 
lem  with  deferring  both  P  and  Q  until  after  time  10.  However,  such  a  decision  would 
lead  to  failure:  if  the  clock  time  reaches  1 1  and  neither  P  nor  Q  has  been  executed, 
then  all  four  solutions  to  the  DTP  will  have  been  eliminated.  Thus,  in  addition  to  the 
information  in  the  ET,  the  dispatcher  must  also  provide  a  second  type  of  information 
to  the  executive.  The  deadline  formula  (DP)  provides  the  executive  with  information 
about  the  next  deadline  that  must  be  met. 

In  the  next  section,  we  explain  how  to  calculate  the  DE,  which  is  more  complicated 
than  computing  the  ET.  Here  we  simply  complete  the  example,  by  illustrating  how  the 
ET  and  the  DE  would  be  updated  as  time  passes.  The  initial  DE  would  indicate  that 
either  P  or  Q  must  be  executed  by  time  10.  Suppose  that  at  time  8,  action  P  is  exe¬ 
cuted.  At  this  point,  STP3  and  STP4  are  no  longer  solutions.  The  ET  then  becomes  { 
<2,  { [15,20]  }>,  <R,  {[11,12],  [21,22]  }>  }  and  the  DE  is  trivially  “2  by  20”  .  In 

this  case,  an  update  to  ET  and  DE  resulted  because  an  activity  occurred.  However, 
updates  may  also  be  required  when  an  activity  does  not  occur  within  an  allowable  time 
window.  Eor  example,  if  R  has  still  not  executed  at  time  13,  then  its  entry  in  the  ET 
should  be  updated  to  be  just  the  singleton  [21,22],  with  no  changes  required  to  the  DE. 
The  example  presented  in  this  section  contains  variables  with  very  little  interaction. 
In  general,  there  can  be  significantly  more  interaction  amongst  the  temporal  con¬ 
straints,  and  the  DE  can  be  arbitrarily  complex. 


4  The  Dispatch  Algorithm 

We  now  sketch  our  algorithm  for  the  dispatch  of  plans  encoded  as  DTPs.  The  in¬ 
put  is  a  DTP  and  the  output  is  an  Execution  Table  (ET)  and  a  Deadline  Eormula  (DE). 
Eor  each  pair  <x,  TW(v)>  in  ET,  v  must  be  executed  some  time  within  TW(v).  It  is  up 
to  the  executive  to  decide  exactly  when.  The  DE  imposes  the  constraint  that  F  has  to 
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hold  by  time  t,  where  a  variable  that  appears  in  the  DF  becomes  true  when  its  corre¬ 
sponding  event  is  executed. 

The  dispatch  algorithm  will  be  called  in  three  circumstances:  (1)  when  a  new  plan 
needs  to  have  its  dispatch  information  initialized,  at  or  before  time  TR\  (2)  when  an 
event  in  the  DTP  is  executed;  (3)  when  an  opportunity  for  execution  passes  because 
the  clock  time  passes  the  upper  bound  of  a  convex  interval  in  the  time  window  for  an 
action  that  has  not  yet  been  executed.  Pseudo-code  is  provided  in  Figure  1.  Space 
constraints  preclude  detailed  description  of  the  algorithm  (but  see  15]).  Here  we  sim¬ 
ply  illustrate  the  procedure  for  computing  the  DF,  the  most  interesting  part  of  the 
algorithm. 

Recall  the  example  above.  Initially,  at  time  TR,  the  DTP  has  four  solutions.  To 
determine  the  initial  DF,  we  consider  the  next  critical  moment,  NC,  which  is  the  next 
time  at  which  any  action  must  be  performed.  This  time  is  equal  to  the  minimal  value 
of  all  the  upper  bounds  on  time  windows  for  actions,  i.e.,  it  is  min{U(x,i)l  x  is  an  ac¬ 
tion  in  the  DTP,  and  i  is  a  solution  STP}.  For  instance,  in  our  example  DTP,  U(P,  1) 
=  U(P,  2)  =  10.  The  actions  that  may  need  to  be  executed  by  NC  are  those  x  such  that 
U(x,i)  =  NC  for  some  STP  i.  We  create  a  list  UMIN  containing  ordered  pairs  <x,i> 
such  that  U(x,i)  =  NC.  In  our  current  example,  UMIN  =  {<P,  1>,  <P,  2>,  <Q,  3>, 
<Q,  4>}.  Now  we  perform  the  interesting  part  of  the  computation.  If  <x,i>  is  in 
UMIN  ,  it  means  that  unless  x  is  executed  by  time  NC,  STPi  will  cease  to  be  a  solu¬ 
tion  for  the  DTP.  It  is  acceptable  for  STPi  to  be  eliminated  from  the  solution  set  only 
if  there  is  at  least  one  alternative  STP  that  is  not  simultaneously  eliminated.  This  is 
exactly  what  the  deadline  formula  ensures:  that  at  the  next  critical  moment,  the  entire 
set  of  solutions  will  not  be  simultaneously  eliminated.  We  thus  use  a  minimal  set 
cover  algorithm  to  compute  all  sets  of  pairs  <x,i>  in  UMIN  such  that  the  i  values 
form  a  minimal  cover  of  the  set  of  solution  STPs.  In  our  example,  there  is  only  one 
minimal  cover,  namely  the  entire  set  UMIN.  Thus,  the  initial  DF  specifies  that  P  or  Q 
must  be  executed  by  time  10:  <Pv  Q,  10>.  In  general,  there  may  be  multiple  mini¬ 
mal  covers  of  the  solution  STPs:  in  that  case,  each  cover  specifies  a  disjunction  of 
actions  that  must  be  performed  by  the  next  critical  time.  For  instance,  suppose  that 
some  DTP  has  four  solution  STPs,  and  that  at  time  TR,  U  (L,  1)  =  U  (L,  2)  =  U  (M,  3) 
=  U  (M,  4)  =  U  (N,  4)  =  U  (S,  3)  =  10.  Then  by  time  10  either  L  or  M  must  be  exe¬ 
cuted;  additionally,  at  least  one  of  L  or  N  or  S  must  be  executed.  The  corresponding 
DF  is  <(Lv  M)a(Lv  Nv  S),  10>. 


5  Formal  Properties  of  the  Algorithm 

The  role  of  a  dispatcher  is  to  notify  the  executive  of  when  actions  may  be  executed 
and  when  they  must  be  executed.  Informally,  we  will  say  that  a  dispatch  algorithm  is 
correct  if,  whenever  the  executive  executes  actions  according  to  the  dispatch  notifica¬ 
tions,  the  performance  of  those  actions  respects  the  temporal  constraints  of  the  under¬ 
lying  plan.  Obviously,  dispatch  algorithms  should  be  correct,  but  correctness  is  not 
enough.  Dispatchers  should  also  be  deadlock-free:  they  should  provide  enough  in¬ 
formation  so  that  the  executive  does  not  violate  a  constraint  through  inaction.  A 
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Initial-Dispatch  (DTP  D) 

1.  Find  all  n  solutions  (consistent  component  STPs)  to  D,  calculate  their  distance 
graphs,  and  store  them  in  Solutions  [i].  Associate  each  solution  with  its  (integer¬ 
valued)  index. 

2.  Set  the  variable  TR  to  have  the  status  Executed,  and  assign  TR=0. 

3 .  Compute-Dispatch-Info(Solutions) . 

Update-for-Executed-Event  (STP  [11  Solutions) 

1 .  Let  V  be  the  event  that  was  just  executed,  at  time  t. 

2.  Remove  from  Solutions  all  STPs  i  for  which  t  ^  TW  (v,/). 

3.  Propagate  the  constraint  t<x-  TR  <  tin  all  remaining  Solutions. 

4.  Mark  x  as  Executed. 

5.  Compute-Dispatch-Info  (Solutions). 

Update-for-Violated-Bounds  (STPfil  Solutions) 

1.  Let  U  =  {U  (x,  k)\  U  (x,  k)  <  Current-Time} 

2.  Remove  from  Solutions  all  STPs  k  that  appear  in  U. 

3.  Compute-Dispatch-Info  (Solutions). 

Compute-Dispatch-Info  (STPfi!  Solutions) 

1 .  For  each  event  v  in  Solutions 

2.  {If  X  is  enabled 

3.  ET  =  ETu<x,  TW(v)>}. 

4.  Let  U  =  the  set  of  upper  bounds  on  time  windows,  \](x,i)  for  each  still  un¬ 
executed  action  x  and  each  STP  i. 

5.  Let  NC,  the  next  critical  time  point,  be  the  value  of  the  minimum  upper 
bound  in  U. 

6.  Let  \Jmin  =  {U(v,  /)l  \](x,i)  =  NC}. 

7.  Eor  each  x  such  that  \](x,i)  g  Cmin^  let  Sx=  {/I  \J(x,i)  g  Um/a^} 

8 .  { Initialize  E  =  true ; 

9.  Eor  each  minimal  solution  MinCover  of  the  set-cover  problem  (Solu¬ 
tions,  let  E  =  E  A  (V  I  MinCover  x). 

10.  DE  =  <E,  AC>.} 


Eigure  1 .  The  Dispatch  Algorithm 

third  desirable  property  for  dispatchers  is  maximal  flexibility:  they  should  not  issue  a 
notification  that  unnecessarily  eliminates  a  possible  execution,  i.e.,  an  execution  that 
respects  the  constraints  of  the  underlying  plan.  Einally,  we  will  require  dispatch  algo¬ 
rithms  to  be  useful,  in  the  sense  that  they  really  do  some  work.  Usefulness  will  be 
defined  as  producing  outputs  that  require  only  polynomial-time  reasoning  on  the  part 
of  the  executive.  Without  a  requirement  of  usefulness,  one  could  achieve  the  other 
three  properties  by  designing  a  DTP  dispatcher  that  simply  passed  the  DTP  represen¬ 
tation  of  a  plan  on  to  the  executive,  letting  it  do  all  the  reasoning  about  when  to  exe¬ 
cute  actions. 


no 
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Our  dispatch  algorithm  has  these  four  properties,  as  proved  in  [5].  The  proofs  de¬ 
pend  on  a  more  precise  notion  of  how  the  dispatcher  and  the  executive  interact.  The 
dispatcher  issues  a  notification  sequence,  a  list  of  pairs  <ET,DF>i  .  .  .  ,<ET,DF>n, 
with  a  new  notification  issued  every  time  an  event  is  executed  or  an  upper  bound  is 
passed.  The  executive  performs  an  execution  sequence,  a  list  Xi=  tj,  ...,  indi¬ 

cating  that  event  Xi  is  executed  at  time  ti,  subject  to  the  restriction  that  j>i  ^  tj  >  tf. 
An  execution  sequence  is  complete  if  it  includes  an  assignment  for  each  event  in  the 
original  DTP;  otherwise  it  is  partial.  The  notification  and  execution  sequences  will  be 
interleaved  in  an  event  sequence.  We  associate  each  execution  event  with  the  preced¬ 
ing  notification,  writing  Notif(v^)  to  denote  the  notification  of  event  Xi. 

Definition.  An  execution  sequence  E  respects  a  notification  sequence  N  iff 

1.  For  each  execution  event  Xi=ti  in  E,  <Xi,  TW  (Xi)>  appears  in  ET  of  Notif  (Xi)  and 
ti  G  TW(V;),  i.e.,  each  event  is  performed  in  its  allowable  time  window. 

2.  For  each  DF=<F,r>  in  N,  {Xf  \Xi  =  tie  E  and  ti  <t}  satisfies  E.  That  is,  the  execu¬ 
tion  sequence  satisfies  all  the  deadline  formulae. 

Theorem  1:  The  dispatch  algorithm  in  Fig.  1  is  correct,  i.e.,  any  complete  execution 
sequence  that  respects  its  notifications  also  satisfies  the  constraints  of  D. 

Theorem  2:  The  dispatch  algorithm  in  Fig.  1  is  deadlock-free,  i.e.,  any  partial  execu¬ 
tion  that  respects  its  notifications  can  be  extended  to  a  complete  execution  that  satis¬ 
fies  the  constraints  of  D. 

Theorem  3:  The  dispatch  algorithm  in  Fig.  1  is  maximally  flexible,  i.e.,  every  com¬ 
plete  execution  sequence  that  respects  the  constraints  in  D  will  be  part  of  some  com¬ 
plete  event  sequence. 

Theorem  4:  The  dispatch  algorithm  in  Fig.  1  is  useful,  i.e.,  generating  an  execution 
sequence  is  polynomial  in  the  size  of  the  notifications. 
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Abstract 

An  important  aspect  of  Business  to  Business  E- 
Commerce  is  the  agile  Virtual  Enterprise  (VE).  VEs  are  es¬ 
tablished  when  existing  enterprises  dynamically  form  tem¬ 
porary  alliances,  joining  their  business  in  order  to  share 
their  costs,  skills  and  resources  in  supporting  certain  activ¬ 
ities.  Currently,  existing  enterprises  use  workflows  to  auto¬ 
mate  their  operation  and  integrate  their  information  systems 
and  human  resources.  Thus,  the  establishment  of  a  VE  has 
been  viewed  as  a  problem  of  dynamically  expanding  and  in¬ 
tegrating  workflows.  In  this  paper,  we  present  an  approach 
to  combining  workflows  from  different  enterprises,  using 
techniques  developed  in  the  Artificial  Intelligence  literature 
on  planning.  Our  method  takes  two  workflow  views,  one 
representing  a  service  request  and  the  other  a  service  provi¬ 
sion  (advertisement),  with  a  mix  of  vital  and  nonvital  steps 
and  a  rich  set  of  constraints,  and  returns  a  list  of  possible 
legal  combinations,  if  any  exist.  It  then  uses  plan-merging 
techniques  to  find  potential  conflicts  between  the  two  work¬ 
flows,  and  to  suggest  additional  constraints  that  can  resolve 
the  conflicts.  The  returned  solutions  represent  terms  for  the 
establishment  of  a  new  VE,  and  can  be  evaluated  by  each 
side  to  determine  which  is  most  desirable. 


*This  material  is  based  upon  work  partially  supported  by  NSF  IIS- 
9812532  and  AFOSR  F30602-00-0547  awards. 

^On  Leave  of  Absence  at  Hewlett-Packard  Laboratories,  MS  lU-17, 
Palo  Alto,  CA  94304,  USA. 


1.  Introduction  and  Motivation 

Electronic  Commerce  is  expanding  from  the  simple  no¬ 
tion  of  E-Store  to  the  notion  of  Virtual  Enterprises  (VEs) 
where  existing  enterprises  dynamically  form  temporary  al¬ 
liances,  joining  their  business  in  order  to  share  their  costs, 
skills  and  resources  in  supporting  certain  activities.  An  ex¬ 
ample  of  a  VE  in  the  context  of  the  travel  industry  would  be 
the  collaboration  of  different  travel  agents,  airliners,  ground 
transportation  services,  hotels,  restaurants  and  entertain¬ 
ment  services  in  order  to  set  up  and  manage  a  tourism  trip. 

Many  enterprises  use  workflows  to  automate  their  oper¬ 
ation  and  integrate  their  information  systems  and  human  re¬ 
sources  [19].  A  workflow  consists  of  a  set  of  activities  (also 
called  tasks)  that  need  to  be  executed  according  to  given 
temporal  constraints  over  a  combination  of  heterogeneous 
database  systems  and  legacy  systems.  A  major  challenge 
has  been  the  development  of  workflow  management  systems 
(e.g.,  [9,  5,  13,  1]).  Several  techniques  have  been  devel¬ 
oped  for  correct  and  reliable  specification,  execution,  and 
monitoring  of  workflows  and  the  involved  external  support. 
Many  of  these  techniques  are  extensions  of  those  in  transac¬ 
tion  processing  in  databases  combined  with  general  middle¬ 
ware  services  such  as  those  found  in  CORBA/DCOM  and 
more  recently  in  Java-based  services  such  as  Jini  from  Sun 
and  E*Speak  from  HR 

Very  recently  the  idea  of  the  use  of  workflows  to  sup¬ 
port  multi-organizational  processes  that  form  a  virtual  en¬ 
terprise  has  attracted  some  attention  [10,  6,  8].  The  estab¬ 
lishment  of  a  VE  can  be  seen  as  a  problem  of  dynamically 
expanding  and  integrating  workflows  in  decentralized,  au¬ 
tonomous  and  interacting  workflow  management  systems 
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[2,  7,  12].  During  the  establishment  of  a  VE,  a  distributed, 
multi-organizational  workflow  emerges  from  the  dynamic 
merging  and  reconfiguration  of  workflows  representing  E- 
Services  in  the  participating  enterprises.  In  our  previous 
work,  we  looked  at  using  mobile  agents  as  a  platform  for 
advertising,  negotiating,  and  exchanging  control  informa¬ 
tion  about  E-Services  for  the  establishment  of  VE’s  [6].  In 
this  paper,  we  focus  on  a  method  for  verifying  that  the  VE  is 
compatible  with  workflows  in  the  participating  enterprises. 

The  contribution  of  this  paper  is  a  new  method  for  es¬ 
tablishing  VEs,  involving  both  the  generation  of  outsourc¬ 
ing  requests  and  the  validation  of  constraints.  The  scheme 
incorporates  techniques  developed  in  the  Artificial  Intelli¬ 
gence  (AI)  literature  on  planning,  specifically  algorithms 
for  merging  temporal  plans.  Within  the  AI  literature,  a  plan 
is  a  collection  of  steps  (i.e.,  tasks),  with  causal,  temporal, 
and  resource  constraints.  A  plan  is  intended  to  represent  a 
course  of  action  that  will  achieve  a  specified  goal  when  ex¬ 
ecuted  beginning  in  a  specified  initial  state.  Critical  to  the 
notion  of  plans  is  that  of  causal  structure:  the  steps  in  each 
plan  are  specified  in  terms  of  their  preconditions  and  effects, 
and  the  plan  records  information  about  which  steps  cause 
(or  establish)  the  preconditions  of  other  steps.  When  merg¬ 
ing  together  two  plans,  it  is  necessary  not  only  to  check  that 
there  are  no  violations  of  the  temporal  and  resource  con¬ 
straints  of  the  plans  being  merged,  but  also  to  ensure  that 
the  necessary  causal  relations  are  maintained  in  the  merged 
plan.  We  argue  in  this  paper  that  similar  consistency  re¬ 
quirements  also  hold  when  a  VE  is  formed. 

In  the  next  section,  we  review  the  basic  structures  used  in 
workflows.  In  particular,  we  describe  a  class  of  workflows 
that  include  specifications  of  preconditions  and  effects.  In 
Section  3,  we  describe  a  VE  and  its  components.  Section  4 
describes  our  detailed  scheme  for  establishing  such  a  VE. 
Section  5  deals  with  the  current  state  of  our  implementation. 
We  conclude  with  a  summary  in  Section  6. 

2.  Workflow  Model 

Workflows  encode  tasks  and  the  relationships  among 
them.  Workflow  specification  formalisms  generally  provide 
a  small  set  of  basic  control  flow  relationships  among  tasks. 
Typically  there  are  four  such  relationships:  OR-split,  AND- 
split,  OR-join  and  AND-join.  The  first  two  relationships  are 
used  to  specify  branching  decisions  in  a  workflow  whereas 
the  remaining  two  specify  points  where  activities  converge 
to  initiate  the  next  activity  within  a  workflow.  An  OR-join 
specifies  alternatives  whereas  AND-join  specifies  required 
activities. 

While  the  relations  just  listed  provide  information  about 
the  relative  ordering  of  the  tasks  in  a  given  workflow,  to 
handle  the  problem  of  forming  Virtual  Enterprises,  it  is  also 
necessary  for  the  workflow  to  model  a  significant  amount 


of  information  about  each  task.  Thus,  we  will  assume  an 
enriched  model  of  workflows  in  which  each  task  has  the 
following  information  associated  with  it: 

•  Pre-  and  postconditions,  which  specify  what  must  be 
true  before  a  task  can  be  executed,  and  what  will  be 
made  true  as  a  result  of  the  task’s  performance. 

•  Causal  links,  which  relate  each  task  that  establishes  a 
condition  (listed  in  its  postconditions)  to  the  task  that 
requires  it  (listed  in  its  preconditions). 

•  In-  and  out-parameters,  which  are  used  in  the  evalu¬ 
ation  of  preconditions  and  postconditions.  They  carry 
information  and  engender  data  flow  during  execution. 
Eor  example,  a  credit  card  number  could  be  an  in¬ 
parameter  to  a  “pay  for  dinner”  task. 

•  Temporal  Constraints  that  specify  the  earliest  and  lat¬ 
est  start  and  end  times  of  a  task,  as  well  as  the  minimal 
and  maximal  durations  of  the  task.  They  can  be  abso¬ 
lute  times  or  relative  to  the  execution  of  other  tasks. 

•  Resource  Constraints,  which  specify  the  equipment, 
material,  or  agent  resources  required  for  the  task. 

•  Significance,  which  indicates  whether  the  task  is  vi¬ 
tal  to  the  workflow  and  therefore  must  be  executed, 
or  whether  it  is  nonvital,  and  need  only  be  executed  if 
feasible  [6]. 

•  Cost,  which  represents  the  price  of  the  task. 

Other  information  may  also  be  associated  with  each  task, 
such  as  rules  for  exception  handling  should  the  task  fail. 
However,  we  will  not  be  concerned  with  these  types  of  in¬ 
formation  in  the  current  paper. 

As  shown  in  Eigure  1,  a  workflow  can  be  graphically 
depicted  with  nodes  (thick  boxes)  denoting  activities  and 
arrows  denoting  precedence.  The  figure  represents  a  busi¬ 
ness  trip  from  [15].  Shaded  nodes  indicate  vital  activities 
that  must  be  completed  to  ensure  proper  execution.  Nodes 
with  a  pair  of  dashed  lines  leading  to  another  workflow  are 
hierarchical  activities:  those  that  can  be  decomposed  into 
workflows  themselves.  AND-splits  and  AND-joins  are  rep¬ 
resented  implicitly  when  two  or  more  causal  links  emanate 
or  arrive  at  a  node  respectively.  To  represent  OR-splits  we 
insert  a  conditional  node  that  creates  two  new  execution 
contexts  (branches),  e.g.,  one  for  success  and  one  for  failure 
(in  Eigure  1,  these  are  shown  as  nodes  with  edges  labeled  S 
and  E).  Tasks  are  executed  only  when  their  context  is  true. 
The  OR-join  is  represented  implicitly  when  the  context  S  or 
E  disappears  from  the  labels  of  subsequent  edges. 

We  are  assuming  a  typical  Workflow  Management  Sys¬ 
tem  (WfMS)  architecture  with  our  enriched  model  of  work¬ 
flows.  Specifically,  a  WfMS  consists  of  the  following  three 
basic  components: 
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Figure  1.  Trip  Plan  Workflow 


•  Workflow  Schema  Library,  which  contains  workflow 
schemas  or  templates  and  generic  constraints. 

•  WfMS  Services,  functions  provided  by  the  WfMS  for 
managing  workflows.  These  include  specifying  work- 
flows,  verifying  their  correctness,  instantiating  and 
scheduling  them,  executing  them,  and  monitoring  their 
execution. 

•  Workflow  Repository,  which  contains  all  instantiated 
and  scheduled  workflows,  i.e.,  the  workflows  the  busi¬ 
ness  is  committed  to  performing. 


3.  Forming  Virtual  Enterprises 

A  Virtual  Enterprise  (VE)  is  formed  when  a  business  de¬ 
cides  to  commit  to  a  new  workflow,  while  outsourcing  some 
of  the  work  involved  in  that  workflow.  Consider  the  exam¬ 
ple  of  Jane  Smith,  an  executive  planning  a  trip  to  Vienna. 
She  gets  in  touch  with  a  travel  agency  to  arrange  the  trip. 
She  decides  that  while  she  is  there  she  would  like  to  attend 
an  opera  and  tour  the  Art  Museum.  This  adds  the  nonvital 
nodes  “buy  opera  ticket”  and  “buy  museum  tour  ticket”  to 
the  trip  schema  (Eigure  1).  The  travel  agency  lacks  connec¬ 
tions  with  the  entertainment/opera  industry,  so  is  unable  to 
purchase  such  tickets.  In  order  to  satisfy  the  customer,  they 
decide  to  outsource  those  tasks. 

The  above  example  represents  a  common  reason  for  out¬ 
sourcing.  When  a  business  receives  a  new  request  from  a 
client,  it  takes  the  form  of  an  instantiated  workflow  schema 
from  its  Workflow  Schema  Library.  The  client  may  have 
added  constraints  and/or  customized  the  schema  by  adding 


new  nodes,  which  could  represent  extra  or  special  activi¬ 
ties  and  opportunities.  The  business  may  select  some  of  the 
tasks  from  the  workflow  to  outsource  and/or  it  may  select 
some  of  the  open  conditions  from  the  workflow  and  out¬ 
source  their  achievement.  This  outsourcing  establishes  a 
VE. 

In  our  VE  workflow  speciflcation,  we  use  the  notion  of 
views  to  express  outsourcing.  Any  subgraph  of  a  workflow 
graph  deflnes  a  segment  or  a  view  of  the  workflow.  Eor- 
mally,  a  workflow  view  can  be  deflned  as  a  projection  on  the 
graph  based  on  some  criteria  {pro jeciion{work flow ^  < 
criteria  >)).  Eor  example,  consider  the  view  that  includes 
all  and  only  the  vital  nodes  of  the  full  workflow.  The  re¬ 
quirement  that  nodes  be  vital  is  the  criteria  used  by  the  pro¬ 
jection. 

VitalView  —  pro jection{work flow ^  {a  \ 
a  G  workflow  A  a.significance  =  vital}) 

The  nodes  in  a  view  retain  all  information  of  their  originals, 
including  all  constraints.  However,  because  all  constraints 
are  maintained,  a  view  may  have  nodes  that  have  temporal 
constraints  referring  to  other  nodes  not  actually  in  the  view, 
and  may  also  have  broken  causal  links  possibly  resulting  in 
unsatisfled  preconditions  (i.e.,  a  node  in  the  view  could  have 
a  precondition  that  was  established  by  some  node  in  the  full 
workflow  that  is  not  in  the  view). 

A  workflow  view  can  represent  any  activity  performed 
by  a  service  provider  on  behalf  of  a  service  requester.  Con¬ 
sequently,  workflow  views  can  be  used  to  express  service 
requests  or  service  provision  (advertisement).  In  our  pro¬ 
posed  system,  it  is  these  workflow  views  that  are  being  re¬ 
quested  and  advertised. 

In  our  scheme,  a  request  has  the  following  structure: 
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Requests:  Rq  =  (P,  G,  RW) 

where  P  =  Service  Requester  Profile, 

G  =  set  of  Goals, 

RW  =  Requested  Workflow  View 

The  profile  can  contain  various  information  about  the  re¬ 
quester,  such  as  name,  site  identification,  credentials,  etc.. 
It  may  also  contain  a  target  price  range.  The  set  of  goals 
is  a  list  of  all  goals  (postconditions)  that  need  to  be  ac¬ 
complished.  The  workflow  view  captures  all  temporal  con¬ 
straints  and  resource  usage  issues  involved. 

In  the  example  above,  the  request  includes  the  profile  of 
the  travel  agency,  the  goals  “opera  ticket  purchased”  and 
“museum  ticket  purchased”,  and  a  view  with  two  nodes  that 
indicate  times  by  which  the  tickets  must  be  purchased. 

A  requested  workflow  view  can  be  potentially  aug¬ 
mented  during  negotiation  to  match  the  service  provider’s 
workflow,  reflecting  opportunities,  omitted  activities  and 
data.  During  a  negotiation  we  may  decompose  the  required 
view  into  several  views  and  seek  other  service  providers  for 
the  other  parts  of  the  view.  In  this  way,  a  single  initial  re¬ 
quest  may  lead  to  the  establishment  of  a  VE  comprising 
multiple  enterprises.  A  VE  comprising  multiple  enterprises 
can  also  result  when  a  service  provider’s  view  includes  out¬ 
sourcing.  We  will  elaborate  on  this  in  the  next  section. 

The  structure  of  an  advertisement  is  the  same  as  that  of  a 
request. 

Advertisements:  Ad  —  (P,  G,  AW) 
where  P  =  Service  Provider  Profile, 

G  =  set  of  Goals, 

AW  =  Advertised  Workflow  View 

It  includes  a  profile,  the  set  of  goals  accomplished,  and  a 
workflow  view  encompassing  constraints.  The  profile,  in 
addition  to  other  information,  may  contain  cost  information 
for  the  workflow  as  a  whole,  such  as  minimum  cost,  maxi¬ 
mum  cost  (cost  with  all  nonvital  steps),  or  both.  The  list  of 
goals  indicates  what  the  advertised  workflow  actually  does, 
and  may  also  include  goals  associated  with  nonvital  activ¬ 
ities.  Such  advertisements  will  typically  be  stored  in  the 
databases  of  trading  servers.  Each  provider  may  have  a  set 
of  advertisements  with  the  same  goals  but  with  a  different 
associated  workflow  view  (i.e.,  different  constraints). 

To  return  to  our  example,  an  advertisement  that  would  be 
of  interest  to  the  travel  agency  would  be  for  a  business  that 
specializes  in  Vienna  cultural  events,  including  opera.  The 
single  goal  “opera  ticket  purchased”  is  accomplished.  Its 
workflow  view  includes  the  tasks  “contact  opera  houses”, 
“read  current  reviews”,  and  “purchase  ticket.” 

The  VE  environment  is  a  distributed  environment.  It 
consists  of  multiple  businesses,  acting  as  requesters  and 
providers,  using  services  provided  by  negotiation  areas  or 
trading  places.  The  trading  places  could  contain  databases 


of  advertisements  and  could  provide  services  allowing  busi¬ 
nesses  to  both  place  advertisements  and  to  find  advertise¬ 
ments  that  meet  their  goals.  Standardization  of  represen¬ 
tation  is  clearly  required  (particularly  of  preconditions,  ef¬ 
fects,  and  goals),  and  could  be  enforced  by  the  trading 
servers.  A  portion  of  this  environment  is  shown  in  Eigure  2. 
Two  negotiation  areas  are  depicted,  as  well  as  four  busi¬ 
nesses’  WfMS.  Shaded  nodes  again  represent  vital  activi¬ 
ties,  and  an  advertised  hierarchical  Opera  activity  is  shown 
partially  expanded. 

3.1.  Commitment  and  Outsourcing  Request 
Generation 

In  order  to  commit  a  new,  possibly  customized  workflow, 
a  WfMS  needs  to  make  sure  that  it  is  schedulable.  A  work- 
flow  is  schedulable  if  it  is  correct,  complete,  and  compatible 
with  existing  commitments. 

Definition  1  Workflow  Correctness:  A  workflow  is  cor¬ 
rect  if  and  only  if 

1.  it  has  no  conflicting  temporal  or  resource  constraints, 

2.  for  each  goal/precondition  P,  there  is  a  task  that 
achieves  P  (the  producer  task),  and  it  is  ordered  be¬ 
fore  the  task  that  requires  it  (the  consumer  task),  and 

2.  for  each  goal/precondition  P,  no  task  that  may  negate 
P  can  possibly  be  ordered  in  between  the  producer  and 
the  consumer. 

This  notion  of  correctness  is  important  as  only  correct 
workflows  can  possibly  be  executed.  Note  that  some  work- 
flows  may  contain  preconditions  that  are  assumed  to  be  es¬ 
tablished  independently  of  the  workflow  itself.  We  will  call 
such  preconditions  open  with  respect  to  the  workflow.  A 
simple  example  of  such  an  open  precondition  is  a  work- 
flow  for  renting  a  car  that  assumes  the  precondition  of  hav¬ 
ing  a  driver’s  license.  Workflows  with  such  open  precondi¬ 
tions  are  incorrect  until  they  have  been  combined  with  other 
workflows  that  establish  all  open  preconditions. 

Definition  2  Workflow  Completeness:  A  complete  work- 
flow  is  a  workflow  that  specifies  all  tasks  needed  to  achieve 
its  goals  and  preconditions. 

Definition  3  Workflow  Compatibility:  A  workflow  is 
compatible  with  another  if  none  of  its  nodes  conflict  with 
any  of  the  other’s  (and  vice  versa). 

This  means  that  the  temporal  constraints,  resource  usage, 
and  postconditions  of  its  nodes  do  not  prevent  the  execution 
of  the  nodes  in  the  other  workflow  (though  they  may  place 
limits  on  when  those  nodes  can  be  executed).  So  for  ex¬ 
ample,  a  compatibility  conflict  between  workflows  arises  if 
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Figure  2.  VE  Environment 


two  tasks  that  use  the  same  resource  (e.g.,  equipment)  are 
set  to  execute  at  the  same  time.  Another  example  is  a  task 
that  dictates  that  a  robot  move  to  the  printing  room  for  the 
purpose  of  getting  a  faxed  itinerary,  which  conflicts  with  a 
task  that  moves  the  robot  to  another  room  that  could  be  ex¬ 
ecuted  after  going  to  the  printing  room  but  before  fetching 
the  fax. 

An  alternative  definition  of  the  compatibility  of  two 
workflows  is  that  the  workflow  resulting  from  their  union 
is  correct.  We  propose  the  notion  of  a  merge  with  the 
Workflow  Repository  for  determining  the  compatibility  of 
a  workflow  with  the  currently  scheduled  workflows  (in  the 
Repository).  If  the  merge  is  successful,  the  new  workflow 
can  be  connnitted  and  its  execution  enabled.  If  the  merge  is 
unsuccessful,  the  new  workflow  is  not  compatible  and  the 
business  may  consider  outsourcing. 

An  effective  merging  process  will  check  whether  the 
above  requirements  (correct,  complete,  and  compatible)  are 
met,  and  will  indicate  where  problems  lie:  what  nodes  are 
conflicting  with  others,  which  have  unsatisfied  (open)  pre¬ 
conditions,  or  which  the  business  lacks  the  necessary  ex¬ 
pertise  (i.e.,  roles  as  resources)  to  accomplish.  It  may  also 
suggest  additional  temporal  or  resource  constraints  that  are 
required  to  ensure  that  they  are  met.  However,  it’s  desirable 
to  impose  a  minimal  set  of  extra  constraints,  i.e.,  to  provide 
a  least-connnitment  response,  as  this  allows  increased  flexi¬ 
bility  to  respond  to  changes  that  may  arise  during  execution. 

The  merge  process  can  also  be  used  to  identify  and  con¬ 
struct  outsourcing  requests.  In  the  event  of  an  unsuccessful 
merge,  any  nodes  from  the  new  view  that  are  indicated  as 
problems  by  the  merging  process  (those  having  irresolvable 


resource  conflicts  with  existing  commitments)  will  form 
part  of  the  requested  workflow  view  VRq  by  extracting  them 
from  the  full  workflow  using  projection.  In  addition  to  these 
nodes,  for  each  open  precondition  in  the  new  view  not  sat¬ 
isfied  by  the  existing  commitments  (such  preconditions  will 
be  found  by  the  merge  process),  a  new  place-holder  node  is 
added  to  the  view.  Each  of  these  new  nodes  represents  a  task 
that  accomplishes  one  of  the  open  preconditions,  i.e.,  it  has 
one  of  the  open  preconditions  set  as  its  postcondition,  and 
any  associated  temporal  and  causal  links  are  applied.  The 
complete  set  of  postconditions  of  every  node  in  VRq  make 
up  the  goal  set  of  the  request,  G.  In  the  simplest  case  VRq 
would  be  a  single  node,  with  associated  constraints.  More 
complex  cases  would  involve  multiple  nodes  and  richer  con¬ 
straints. 

Recall  that  projected  nodes  maintain  all  constraints  and 
conditions  they  had  in  the  parent  workflow,  and  may  there¬ 
fore  include  unsatisfied  preconditions  and  temporal  con¬ 
straints  referring  to  nodes  not  in  the  view.  This  is  not  really 
a  problem  as  they  will  be  satisfied  by  non-outsourced  nodes. 
The  preconditions,  along  with  in-parameters,  represent  the 
input  to  the  outsourced  view.  Goals  and  out-parameters  of 
the  outsourced  nodes  represent  the  output. 

4.  Outsourcing  Scheme 

In  this  section,  we  discuss  in  detail  the  steps  for  outsourc¬ 
ing  and  establishing  a  VE. 

Let  7?  be  a  Requester  and  P  be  a  set  of  providers 
{Pi, Pn}.  R  has  a  set  of  workflows  to  which  it  is  al¬ 
ready  committed,  and  which  it  stores  in  the  WE  Repository; 


116 


let  us  call  them  CR  (commitment  workflows  at  requester). 
Similarly,  each  Pi  has  a  set  of  workflows  already  committed 
to;  let  us  call  them  CP{i). 

Let  Rq  —  (7?,  G^VRq)  be  a  request  of  R  for  outsourcing 
with  goals  G  —  {G*,  ...Gn}  and  workflow  view  VRq.  Each 
Pi  can  provide  a  set  of  alternative  workflow  views  A{Pi) 
for  achieving  one  or  more  Gj  of  Rq. 

The  problem  of  outsourcing  is  how  to  pick  a  set  of  work- 
flow  views  S  from  the  A{Pi)  of  one  or  more  Pi  so  that  the 
combined  set  satisfles  Rq  and  merges  with  CR,  and  each 
A{Pi)  in  it  merges  with  its  provider’s  GP{i).  Speciflcally, 
such  a  set  achieves  all  goals  of  the  outsourced  workflow,  all 
temporal  constraints  are  satisflable,  there  are  no  resource 
conflicts,  and  for  every  precondition  of  every  workflow  ac¬ 
tivity  in  CR  and  CP{i)  there  exists  a  causal  dependency  that 
ensures  that  the  precondition  will  be  met. 

Formally,  we  want  a  set  5  =  wfi  U  wf2  U  wfs  U...U 
u;/n,  where  wfj  G  Ui  A{Pi),  j  =  1, . . . ,  n  such  that 

•  postcondition s{S)  D  G 

(all  outsourcing  goals  are  met) 

•  G  S  postconditions{w fx)  f]  G  ^  <f) 

(each  workflow  achieves  at  least  one  goal) 

•  compatible's,  GR) 

(S  is  compatible  with  the  requester’s  commitments) 

•  G  A{Py)  compatible{wf^,GP{y)) 

(each  alternative  workflow  is  compatible  with  its 
provider’s  commitments) 

The  above  suggests  a  solution  that  has  three  phases: 

1 .  Finding  a  set  of  alternative  workflows  that  satisfy  Rq 
{Terms  for  the  Establishment  of  a  VE) 

2.  Checking  for  the  satisfaction  of  GP{i) 

{Providers  Validation  of  Terms  and  E-Service  Bids) 

3.  Check  for  the  satisfaction  of  R 
{E-Service  Bid  Evaluation) 

We  elaborate  on  these  phases  in  the  next  subsections. 

4.1.  Phase  1:  Terms  for  the  Establishment  of  a  VE 

As  mentioned  previously,  we  assume  in  this  paper  that 
flnding  alternative  workflow  views  that  satisfy  a  request  Rq 
is  a  service  provided  by  trading  servers.  Each  alternative 
view  represents  a  term  for  the  establishment  of  a  VE.  Dur¬ 
ing  this  first  phase  the  sets  A{Pi)  of  alternative  workflow 
views  are  generated.  These  views  accomplish  the  goals  G 
of  Rq  while  not  violating  any  of  its  constraints.  For  the  sake 
of  simplicity,  we  will  assume  in  the  rest  of  our  discussion 
that  there  is  only  one  trading  server. 

The  service  searches  the  database  of  the  trading  server, 
looking  for  advertisements  that  meet  some  or  all  of  the  re¬ 
quested  goals.  Which  advertisements  are  examined  first  de¬ 
pends  on  the  selection  conditions  being  used.  One  such 


condition  would  include  the  desirability  of  first  consider¬ 
ing  those  that  accomplish  all  goals,  and  only  considering 
multiple,  partial  matches  when  all  such  are  found.  For  each 
advertisement  found  and  selected,  the  server  must  finally 
determine  if  it  or  any  of  its  alternatives  (involving  different 
combinations  of  nonvital  nodes)  can  meet  the  constraints  of 
the  request.  This  process  continues  until  all  advertisements 
that  meet  any  goals  have  been  examined,  or  some  termina¬ 
tion  criteria  are  met  (such  as  a  deadline  for  search  time). 

For  the  detailed  explanation,  we  will  consider  one  such 
advertisement  found  and  selected  by  the  trading  server  that 
accomplishes  all  goals  in  G;  let  us  call  it  Adi. 

As  shown  in  Figure  3,  the  service  must  determine  if  Adi 
will  satisfy  the  constraints  in  the  request’s  workflow  view 
Rq.  To  do  this,  Adi  and  Rq  are  first  stripped  down  to  only 
vital  nodes  using  projection.  Temporal  constraints  of  the  vi¬ 
tal  nodes  may  need  to  be  adjusted,  as  any  referring  to  non¬ 
vital  nodes  will  be  invalid.  For  any  node  that  has  such  a 
constraint,  there  are  four  possible  situations: 

1 .  The  nonvital  node  referred  to  has  no  constraints  on  its 
time^  :  the  constraint  on  the  vital  node  can  be  dropped. 

2.  The  nonvital  node  has  an  absolute  time:  that  time  can 
be  used. 

3.  The  nonvital  node  has  a  time  relative  to  some  other 
vital  node:  the  reference  to  that  node  can  be  used. 

4.  The  nonvital  node  has  a  time  relative  to  some  other 
nonvital  node:  that  nonvital  must  be  searched  in  the 
same  fashion  for  a  time  or  vital  node  reference. 

It  may  be  beneficial  to  instead  store  such  alternative  con¬ 
straints  with  the  vital  nodes  in  order  to  save  computational 
time,  though  the  number  of  nonvital  nodes  is  likely  to  be 
small. 

Next  the  service  attempts  to  bind  the  constraints  of  the 
vital-only  view  of  Rq  (called  RqV  in  the  figure)  to  the 
stripped  view  of  Adi  {AdV  in  the  figure).  Binding  adds 
the  constraints  of  the  requested  nodes  to  the  corresponding 
nodes  in  the  advertisement  (those  that  have  the  same  post¬ 
conditions).  If  AdV  cannot  support  the  added  constraints 
(because  they  conflict  with  existing  ones),  the  bind  fails  and 
the  function  must  backtrack  to  find  a  different  advertise¬ 
ment.  Otherwise,  the  new  bound  advertised  view  BV  is 
added  to  A{Pi),  where  Pi  is  the  provider  of  BV,  and  the 
search  continues  for  its  variants  that  include  nonvital  nodes. 

The  search  for  variants  of  BV  considers  combinations  of 
BV  and  nonvital  nodes  from  the  full  Adi  and  Rq.  This  can 
be  achieved  by  the  function  addNodes,  that  adds  a  group  of 
nodes  to  the  workflow  BV,  restoring  any  modified  temporal 
constraints  that  referred  to  them.  This  is  basically  a  merge 
process.  The  addNodes  function  fails  and  returns  null  if  the 

^  By  “time”  we  mean  either  start  or  end  time  of  the  task,  depending  on 
which  the  specific  temporal  constraint  refers  to. 
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resulting  view  is  incorrect  (i.e.,  the  new  nodes  cannot  be 
added  without  violating  constraints).  If  the  function  does 
not  fail,  the  resulting  view  is  added  to  A{Pi). 

It  is  interesting  to  point  out  that  there  is  another  possible 
method  for  this  search:  working  in  the  other  direction,  start¬ 
ing  by  adding  back  all  nonvitals  and  then  removing  them  to 
find  correct  alternatives.  It  is  not  clear  which  approach  is 
better,  but  we  intend  to  investigate  this  in  future  work. 

In  either  case,  the  search  will  proceed  until  all  possible 
combinations  have  been  attempted  or  some  other  termina¬ 
tion  criteria  have  been  reached.  As  most  views  are  expected 
to  have  3  or  fewer  nonvital  steps,  finding  all  possible  com¬ 
binations  is  not  likely  to  be  impractical.  Once  the  search 
has  finished,  A{Pi)  contains  every  alternative  workflow  so¬ 
lution  for  the  workflow  Adi. 

The  service  generates  a  set  A{Pi)  for  each  Pi  with  at 
least  one  selected  advertisement.  The  A(i3)’s  created  in 
this  fashion  are  now  sent  out  to  their  respective  provider  for 
validation. 

4.2.  Phase  2:  Providers  Validation  of  Terms  and 

E-Service  Bids 

The  second  phase  of  the  outsourcing  takes  place  at  the 
providers  of  the  advertisements.  Each  Pi  receives  the  A{Pi) 
generated  for  it  in  the  previous  phase,  and  must  determine 
whether  any  of  the  workflow  views  in  A{Pi)  are  compatible 
with  its  CP{i).  Each  alternative  basically  represents  a  po¬ 
tential  new  incoming  workflow  to  be  scheduled.  Recall  that 
such  scheduling  can  be  accomplished  using  the  merge  pro¬ 
cess.  Thus,  the  provider  attempts  to  merge  each  alternative 
with  CP{i)  independently.  Any  that  fail  are  removed  from 
A{Pi).  Those  that  succeed  can  be  kept  to  form  the  basis  for 
the  service  bid.  Of  course,  if  A{Pi)  is  empty  at  the  end  of 
this  phase,  then  none  of  the  views  were  compatible  with  the 
provider’s  commitments. 

To  generate  the  full  service  bid,  each  view  remaining  in 
A{Pi)  could  possibly  be  expanded  into  multiple  views  if 
the  provider  wishes  to  add  additional  nonvital  nodes  (repre¬ 
senting  special  offers  or  bonuses).  Note  that  such  additions 
would  likely  increase  cost,  but  would  possibly  also  increase 
value.  The  provider  may  also  rank  the  solutions  in  order  of 
preference  or  cost  to  help  the  later  decision  process. 

Each  provider  sends  its  service  bid  to  the  requester  to  be 
evaluated  in  the  next  phase. 

4.3.  Phase  3:  E-Service  Bid  Evaluation 

In  the  third  phase,  the  requester  evaluates  and  selects  an 
E-Service  bid.  Of  the  views  in  the  service  bids  returned 
by  the  providers,  the  requester  must  determine  which  are 
compatible  with  its  CR.  This  is  done  in  exactly  the  same 
fashion  as  with  the  providers. 


Each  service  bid  in  the  returned  list  is  combined  with  the 
rest  of  the  original  workflow  to  form  a  complete  solution. 
Eor  each  solution,  a  merge  is  attempted  with  the  connnit- 
ted  workflows.  Any  that  fail  to  merge  are  discarded.  Those 
that  successfully  merge  are  correct  views  that  each  accom¬ 
plish  the  outsourcing  and  that  are  compatible  with  both  the 
provider’s  and  requester’s  previous  commitments. 

The  requester  may  then  evaluate  these  remaining  views 
to  make  a  final  decision  as  to  which  one  will  be  used,  which 
likely  involves  cost  comparisons. 

4.4.  Multiple  Partial-Solution  Views 

In  the  previous  discussion,  we  assumed  the  simplest  case 
where  there  exist  advertised  views  that  accomplish  all  the 
goals  of  the  request.  However,  in  many  cases  there  may 
be  no  single  advertisements  that  accomplish  them  all.  This 
would  require  views  from  multiple  advertisements  to  be 
combined  in  order  to  meet  the  requester’s  needs.  In  order  to 
handle  these  cases,  the  described  first  and  third  phases  need 
to  be  enhanced. 

Eor  example,  in  Phase  1,  the  search  for  alternatives  must 
also  search  for  combinations  of  advertisements  that  accom¬ 
plish  all  goals.  The  merge  process  can  be  used  again  to  ver¬ 
ify  that  these  combinations  of  advertisements  are  compat¬ 
ible  with  each  other  in  addition  to  meeting  the  constraints 
of  the  request.  Eor  combined  views  belonging  to  a  sin¬ 
gle  provider,  the  combination  (and  its  alternatives  involving 
nonvital  nodes)  are  grouped  together  as  a  single  view. 

The  requester  in  Phase  3  must  be  aware  that  returned 
views  do  not  necessarily  accomplish  all  goals.  Any  E- 
Service  bids  that  only  satisfy  some  of  the  goals  must  be 
combined  with  other  returned  views  to  form  complete  solu¬ 
tions. 

5.  Implementation 

In  our  previous  work,  we  proposed  to  use  mobile  agents 
as  a  platform  for  establishing  VE’s  [6].  Our  goal  is  to  im¬ 
plement  our  scheme  described  in  the  previous  section  on 
this  platform.  The  idea  is  to  use  mobile  agents  to  per¬ 
form  the  phases  of  the  scheme.  The  requester  dispatches  an 
agent  with  its  request.  The  agent  visits  trading  servers,  and 
spawns  copies  of  itself  to  deliver  alternatives  to  different 
providers.  It  then  gathers  all  returned  service  bids  together 
and  delivers  the  results  back  to  the  requester. 

A  core  concept  in  our  scheme  for  integrating  E-Services 
is  the  merge  process.  It  is  this  process  that  verifies  whether 
or  not  different  workflow  views  are  compatible  with  each 
other.  It  is  also  responsible  for  adding  nonvital  nodes  to 
views  and  verifying  that  a  view  is  compatible  with  a  busi¬ 
ness’  existing  workflow  repository.  The  merge  process  can 
even  be  used  to  generate  the  outsourcing  requests. 
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Repeat 

•  Adi  —  Ad  G  DB  \  <  selection  conditions  > 

•  AdV  —  projection{Adl^  {a  |  a  G  Adi  A  a.significance  =  vital}) 

•  RqV  =  projection{Rq^  {a  |  a  G  Rq  A  a.significance  = 

•  Repeat 

-  Nodes  —  projection{Adl^  selected  nonvital  G  Adi)  U  projection{Rq^  selected  nonvital  G  Rq) 

-  Bx  —  addNodes{BV.,  Nodes) 

-  IfBx  null  A{Pi)  =  ^(^0  U  Bx 

•  Until  all  combinations  of  nonvitals  found  or  termination  criteria  met 

Until  all  Ads  found  that  meet  <  selection  conditions  >  or  termination  criteria  met 


Figure  3.  Service  For  VE  Terms 


Merging  is  not  a  trivial  problem.  It  can  be  formulated 
as  a  Constraint  Satisfaction  Problem  or  CSP,  with  tem¬ 
poral  features.  The  process  must  consider  temporal  con¬ 
straints,  resource  usage,  and  causal  links  (preconditions  and 
effects).  There  has  been  a  great  deal  of  research  done  on 
similar  problems  by  the  Artificial  Intelligence  community 
[14,  17,  20].  A  number  of  formalizations  have  been  de¬ 
veloped  for  variations  with  more  or  less  expressivity.  The 
two  that  most  closely  match  our  problem  are  the  Disjunctive 
Temporal  Problem  (DTP)  and  the  Conditional  Disjunctive 
Temporal  Problem  (CDTP). 

For  solving  DTPs  we  have  developed  and  implemented 
a  new  algorithm  called  Epilitis  [16],  along  with  algorithms 
that  convert  CDTPs  to  DTPs  so  that  they  may  be  solved  by 
it  as  well.  Epilitis  builds  on  plan  merging  techniques  used 
in  a  tool  called  PM  A  (Plan  Management  Agent)  [18].  Epili¬ 
tis  integrates  a  number  of  techniques  for  pruning  the  search 
space,  some  of  which  are  Conflict  Directed  Backjumping, 
Removal  of  Subsumed  Variables,  Semantic  Branching,  and 
no-good  learning.  Epilitis  is  currently  the  most  efficient  al¬ 
gorithm  for  solving  such  problems,  as  experimental  results 
have  shown  that  it  is  two  orders  of  magnitude  faster  than 
the  previous  state-of-the-art  solver,  on  synthetic  benchmark 
problems. 

In  the  prototype  system  we  are  currently  developing,  we 
will  use  Epilitis  for  the  merging  process  at  the  WfMS  and 
the  trading  servers.  The  representation  that  Epilitis  expects 
is  nearly  identical  to  our  enhanced  model  of  workflows;  the 
mapping  between  the  two  is  trivial.  Merging  with  Epilitis 
has  all  the  properties  discussed  in  Section  3.  Any  conflicting 
tasks  are  identified  with  explanation,  and  a  minimal  number 


of  constraints  are  added.  Plans  with  disjunctive  temporal 
constraints  are  supported  (for  added  flexibility),  and  dupli¬ 
cate  nodes  can  be  identified  and  pruned/combined. 

Epilitis  does  not  support  the  notion  of  significance  (vi¬ 
tal  vs.  nonvital  tasks).  However  these  are  implemented 
in  the  higher-level  layer  that  performs  the  phases  of  our 
scheme.  Only  this  layer  is  aware  of  the  vitaPnonvital  dis¬ 
tinction.  (This  is  the  cause  of  some  of  the  complexity  in 
the  search  for  alternatives,  as  all  the  different  combinations 
of  nonvitals  must  be  attempted  separately.)  This  layer  also 
serves  to  interface  Epilitis  with  a  relational  DBMS  using 
Microsoft  Access  and  MySQL  that  will  be  used  to  imple¬ 
ment  the  Workflow  Repositories. 

The  current  version  of  Epilitis  is  written  in  LISP,  but  us¬ 
ing  JLinker  we  have  interfaced  it  to  the  rest  of  our  prototype 
which  is  being  developed  in  Java.  The  new  version  of  Epili¬ 
tis  currently  being  developed  will  be  in  Java  as  well. 

6.  Conclusions 

We  are  concerned  with  integrating  E- Services  for  the  es¬ 
tablishment  of  a  VE,  where  such  services  are  represented 
with  workflows.  We  have  therefore  created  algorithms  that 
make  use  of  existing  plan  merging  and  temporal  reasoning 
algorithms  from  the  AI  literature.  Our  scheme  is  sound,  in 
that  the  workflows  it  returns  as  possible  merge  candidates 
are  guaranteed  to  be  correct.  It  is  also  complete,  in  that  it 
will  find  all  such  candidates,  given  sufficient  time.  It  further 
ensures  that  the  merge  candidates  are  compatible  with  all 
businesses  involved  in  the  VE.  It  can  create  the  outsourcing 
requests  based  on  identified  conflicts,  handle  any  number  of 
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nodes  and  workflows  to  be  outsourced,  and  is  flexible  in  that 
it  can  build  a  VE  using  multiple  providers,  each  with  their 
own  set  of  constraints.  Our  scheme  also  takes  into  consider¬ 
ation  that  workflows  have  both  vital  and  nonvital  steps,  and 
appropriately  considers  them  in  its  search. 

In  our  proposed  system,  the  merging  process  is  built  with 
existing  AI  algorithms.  The  speciflc  algorithm,  Epilitis,  is 
the  best  algorithm  available  at  this  time.  It  has  been  imple¬ 
mented  and  is  a  fully  functional  and  working  plan  merging 
tool.  Currently  we  are  developing  our  prototype  system. 
Our  goal  is  to  evaluate  its  performance  in  terms  of  speed 
and  memory  usage.  Another  area  we  intend  to  explore  is 
its  use  as  a  plan/workflow  repair  system  that  would  replace 
broken  or  invalidated  nodes  or  views  with  alternatives,  pos¬ 
sibly  located  in  different  databases  on  various  machines. 

Recently,  there  have  been  a  variety  of  platforms  devel¬ 
oped  with  business  to  business  E-services  and  Virtual  En¬ 
terprises  in  mind.  E*speak  [3]  from  HP,  VorteXML  [4],  and 
CrossElow  [11]  are  examples.  These  systems  provide  vari¬ 
ous  features  for  managing  and  monitoring  VE’s,  along  with 
some  standards  for  communication.  Such  systems  could 
potentially  be  augmented  or  used  conjunctively  with  our 
scheme  for  automated  VE  establishment.  We  will  investi¬ 
gate  such  possibilities  as  part  of  our  future  work. 
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